How can i gather lots of files from one filetype?

43 views Asked by At

Im trying to fuzz some tools but i need a huge amount of .zip or .jpg files for that. I ve tried crawlers like webripper but its not very effective (or im doing it wrong). Is there a better way to get lots of different files?

1

There are 1 answers

0
Ki_Netic On

Ok, for the offchance that someone else might need sth like this:

In the end i used Webripper and instead of generating links to google/bing results with the "filetype" parameter i just put some upload/freeware pages as targeted rip job with the max link depth.

Webripper might crash sometimes and it will take quite some time but well it works somewhat.

A possible better solution would probably be to use the google API (e.g. c#SearchAPI ). Then extract the clean links from the results and call asynch download for those. Using the direct result link most likely wont work because google will block it after some files "Unusual datatransfer".