urllib2覚え書き

PythonでURLを開くためのライブラリ、urllib2の基本的な使い方。

import urllib2
import cookielib

# HTTPをデバッグするよ
hh = urllib2.HTTPHandler(debuglevel=1)

# cookie使うよ
cj = cookielib.Cookiejar()
ch = urllib2.HTTPCookieProcessor(cj)

# HandlerからOpenDirectorのインスタンスを作るよ
opener = urllib2.build_opener(hh, ch)
# リクエストを投げます。
response = opener.open("http://www.google.com")

HTTPのデバッグメッセージが表示されます。

send: 'GET / HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n'
reply: 'HTTP/1.1 302 Found\r\n'
header: Location: http://www.google.co.jp/
header: Cache-Control: private
header: Content-Type: text/html; charset=UTF-8
header: Set-Cookie: PREF=ID=55d8e36ff2405b35:TM=1267881221:LM=1267881221:S=s-dUz3PaXx2rz40i; expires=Mon, 05-Mar-2012 13:13:41 GMT; path=/; domain=.google.com
header: Set-Cookie: NID=32=uP3EE_427CJw3vwUzgLNoAvxq2-rIeUY6aOIx_OwsOunNgldmmy6wUt0Pf7H4vJY4M9n93u_vJOjZXSNgBEY72j7cOptu9TyhnECS8tBkCcV0HQHjO_QZV-XGN-H0j9N; expires=Sun, 05-Sep-2010 13:13:41 GMT; path=/; domain=.google.com; HttpOnly
header: Date: Sat, 06 Mar 2010 13:13:41 GMT
header: Server: gws
header: Content-Length: 221
header: X-XSS-Protection: 0
header: Connection: close
send: 'GET / HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.co.jp\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sat, 06 Mar 2010 13:13:41 GMT
header: Expires: -1
header: Cache-Control: private, max-age=0
header: Content-Type: text/html; charset=Shift_JIS
header: Set-Cookie: PREF=ID=9a0ca1a13152625e:TM=1267881221:LM=1267881221:S=pUQco5Mj4NSXKzgF; expires=Mon, 05-Mar-2012 13:13:41 GMT; path=/; domain=.google.co.jp
header: Set-Cookie: NID=32=M4YEeDAGj6VRt7TukiVq4lcQgsizbAvFjtK_9BXq7VB_1N-NsEy5jOvIhGBn83NPV6gHiVSTN5wUxQzZugbSyH11g-mUH1aGMZRxgsD2f_yl2fYxtkU6BzjiP-WFt24q; expires=Sun, 05-Sep-2010 13:13:41 GMT; path=/; domain=.google.co.jp; HttpOnly
header: Server: gws
header: X-XSS-Protection: 0
header: Transfer-Encoding: chunked
header: Connection: close

レスポンスを確認。

print response.read()