So, I have a scraper that uses bs4 to fetch some links out and then I am willing to download those links but the problem is, when I click on the download button my browser automatically starts to download the zip file that the link points at indirectly, but when I give the same link to urllib, it always ends up downloading the page instead of the zip file!
The link I scrape out is:Link I got from scraping
Link to the page that I scrapped for that link: Main Page
On this main page that link has a text "4931x"
Now when I click the same link through the browser it somehow redirects to:Redirected link
But even when I use urlopen with the 3rd link I get redirected again to the first one!
I have a feeling that this is a stupid question and might have a really easy solution, but I really can't seem to work my way around it. I just want to automatically download the zipped subtitle though urlopen but I can't!
As the network panel of your web browser development tools will show, the original URL returns an HTTP 302 response with a location header that redirects to https://www.flixtools.com/en/osflixtools.subtitles-download/subtitles/6647724?source=source_os_landing&campaign=campaign_sub_landing . That page contains this HTML:
<meta http-equiv="refresh" content="5; URL=//dl.opensubtitles.org/en/download/vrf-108d030f/sub/6647724">. This instructs your web browser to navigate to that URL after waiting for 5 seconds.