unable to deploy portia spider with scrapyd-deploy

Question

Could you please help me figure out what I'm doing wrong ? Here are the steps:

• followed the portia install manual found here https://github.com/scrapinghub/portia - all ok
• created a new project, entered an url, tagged an item - all ok
• clicked "continue browsing", browsed through site, items were being extracted as expected - all ok

Next I wanted to deploy my spider:

1st try : I tried to run, as the docs specified, scrapyd-deploy your_scrapyd_target -p project_name - got error - scrapyd wasn't installed fix: pip install scrapyd

2nd try : I launched scrapyd server, accessed http://localhost:6800/ -all ok

After a brief reading of scrapyd docs I found out I had to edit the file scrapy.cfg from my project : slyd/data/projects/new_project/scrapy.cfg added the following :

[deploy:local]
url = http://localhost:6800/

Went back to the console, checked all is ok :

$:> scrapyd-deploy -l
local http://localhost:6800/

$:> scrapyd-deploy -L local
default

Seemed ok so i gave it another try :

$scrapyd-deploy local -p default
Packing version 1418722113
Deploying to project "default" in http://localhost:6800/addversion.json
Server response (200):
{"status": "error", "message": "IOError: [Errno 21] Is a directory: '/Users/Mike/www/portia/slyd/data/projects/new_project'"}

What am I missing ?

Answer 1

For anyone who stumbles upon this issue, the fix is to deploy scrapyd in another directory other than that of the project. See details here : https://github.com/scrapinghub/portia/issues/128