I'm working on a Scrapy project and I'm trying to deploy it to Scrapy Cloud (by Zyte, formerly known as Scrapinghub). However, during the deployment process, I encounter an error:
[INFO] Deploying dependencies to Scrapy Cloud...
[ERROR] shub.image.push.PushFailed: The referer header is missing. Enable sending it in your browser.
A few things to note:
- I am using the shub deploy command to deploy the project.
- My project structure is standard and I have a settings.py file, which is a Python file.
- I'm not intentionally disabling the 'Referer' headers in my requests.
I have looked into the possibility of browser configurations, VPNs, or other security tools causing this, but I haven't been able to identify the root of the issue.
Has anyone encountered this problem before or can offer any insights into what might be going wrong and how to resolve it?