Multithreaded Downloading Through Proxies In Python

1.1k views Asked by At

What would be the best library for multithreaded harvesting/downloading with multiple proxy support? I've looked at Tkinter, it looks good but there are so many, does anyone have a specific recommendation? Many thanks!

3

There are 3 answers

0
anthony On
2
Kylotan On

Is this something you can't just do by passing a URL to newly spawned threads and calling urllib2.urlopen in each one, or is there a more specific requirement?

2
twneale On

Also take a look at http://scrapy.org/, which is a scraping framework built on top of twisted.