Multithreaded Downloading Through Proxies In Python

Question

Multithreaded Downloading Through Proxies In Python

1.1k views Asked by Cookies At 20 October 2009 at 20:27

What would be the best library for multithreaded harvesting/downloading with multiple proxy support? I've looked at Tkinter, it looks good but there are so many, does anyone have a specific recommendation? Many thanks!

Original Q&A

There are 3 answers

**anthony** · Answer 1 · 2009-10-20 20:28:57

anthony On 20 October 2009 at 20:28

Twisted

**Kylotan** · Answer 2 · 2009-10-20 20:36:02

Kylotan On 20 October 2009 at 20:36

Is this something you can't just do by passing a URL to newly spawned threads and calling urllib2.urlopen in each one, or is there a more specific requirement?

**twneale** · Answer 3 · 2009-10-20 21:24:04

twneale On 20 October 2009 at 21:24

Also take a look at http://scrapy.org/, which is a scraping framework built on top of twisted.

TechQA.

Multithreaded Downloading Through Proxies In Python

There are 3 answers

Related Questions in PYTHON

Related Questions in PROXY

Related Questions in DOWNLOAD

Related Questions in MULTITHREADING

Related Questions in HARVEST

Popular Questions

Popular Tags

Trending Questions