Lots of people may got confused by the Python modules-
httplib, httplib2, urllib, urllib2. Judge from their names, we may prefer the x2 module because they may be version 2 of the module and should work better. Maybe you are right for some degree(as for the httplib).
However, things are a little bit complicated which needs a brief explanation indeed.
What’s the claim
First, let’s see what a specified module’s aim. I will quote from the official site:–)
Here is a link from
The Python Standard Library for
This module defines classes which implement the client side of the HTTP and HTTPS protocols. It is normally not used directly — the module urllib uses it to handle URLs that use HTTP and HTTPS.
This is the official website hosted on
Google Code. It claims:
A comprehensive HTTP client library that supports many features left out of other HTTP libraries.
To become a worthy addition to the standard Python library.
HTTP and HTTPS,
Authentication(support Digest, Basic, WSSE, HMAC Digest and Google Account Authentication),
Lost update support and
Quote from here:
This module provides a high-level interface for fetching data across the World Wide Web. In particular, the urlopen() function is similar to the built-in function open(), but accepts Universal Resource Locators (URLs) instead of filenames. Some restrictions apply — it can only open URLs for reading, and no seek operations are available.
The Standard Python Library stats that:
The urllib2 module defines functions and classes which help in opening URLs (mostly HTTP) in a complex world — basic and digest authentication, redirections, cookies and more. It's an extensible library for opening URLs.
What’s the difference
httplibx and urllibx
urllib/urllib2 is built on top of
httplib. It offers more features than writing to
httplib gives you finer control over the underlying connections.
If you’re dealing solely with HTTP/HTTPs and need access to HTTP specific stuff, use httplib.
For all other cases, use urllibx(note that
urllib and urllib2 have different capabilities, thus are always used together).
httplib and httplib2
Basically, httplib2 is
Google's python httplib implementation but much more powerful.
They actually do the same things utilizing HTTP/HTTPs and other network protocals like FTP, typically we can call them
HTTP client library.
httplib can’t fulfill your need(as if you need the
Redirects feature), consider switching to
To make things simpler, I’d rather using
httplib2 as my default
HTTP client library for Python.
urllib and urllib2
Quote from this post Python: difference between urllib and urllib2:
urllib2 are both Python modules that do URL request related stuff but offer different functionalities. Their two most significant differences are listed below:
urllib2can accept a Request object to set the headers for a URL request, urllib accepts only a URL. That means, you cannot masquerade your User Agent string etc.
urllibprovides the urlencode method which is used for the generation of GET query strings, urllib2 doesn’t have such a function. This is one of the reasons why urllib is often used along with urllib2.
The urllib module has been split into parts and renamed in Python 3 to
urllib.error. The 2to3 tool will automatically adapt imports when converting your sources to Python 3. Also note that the urllib.urlopen() function has been removed in Python 3 in favor of urllib2.urlopen().
- HOWTO Fetch Internet Resources Using urllib2
- How to use urllib2 in Python
- python 模块：httplib、urllib和urllib2的简单用法
- Python 标准库 urllib2 的使用细节
- httplib2 v0.4 documentation