The Duke URLGrabber package for Python makes it incredibly easy to retrieve files from remote servers, and it abstracts urllib2 for you in a protocol-independent way, so you can focus on your application instead of spending time working with Python’s built-in urllib2.
On Windows and Mac OSX, by default, urllib2 (and therefore URLGrabber) will use the built-in proxy settings of the system – but sometimes you don’t want that. For example, let’s say you’re using a Windows box that’s on a domain, and proxy settings have been pushed down by group policy. You want to access local network resources without needing to route through the proxy or authenticate to it.
With urllib2, you would simply add code like the following:
proxy_support = urllib2.ProxyHandler({})
Note the empty dictionary “{}” to specify no proxies. Doing the same for URLGrabber was suggested on a mailing list post – and it apparently worked for the poster. However, I wasn’t able to get it to work, so I came up with another way. I just specified a dummy proxy for a dummy protocol, as follows:
kwargs["proxies"] = { 'nothing': 'http://nothing' }
Have fun!