I use Scrapyd for run my spider dynamically. I add .txt
file that has a list of block words. My problem is following:
When I run Scrapyd server as a daemon it raised the error during scrapping:
NotADirectoryError: [Errno 20] Not a directory: '/tmp/exa-1504173770-gm023ynt.egg/exa/classificator/large.txt'
But if I run Scrapyd server from project dir, all works fine
Here setup.py
code
from setuptools import setup, find_packages
setup(
name='project',
version='1.0',
packages=find_packages() + ['exa'],
entry_points={'scrapy': ['settings = exa.settings']},
package_dir={'exa': 'exa'},
package_data={'exa': ['classificator/large.txt']}
)
And here I'm loading file:
file_dict = open(file_name_dictionary, "r")
self.correct_words = set()
for word in file_dict:
self.correct_words.add(word[:-1])
UPDATE: I fixed this issues, need to use pkg_resources.resource_stream(resource_package, resource_path) for loading file from .egg file
I fixed this issues, need to use pkg_resources.resource_stream(resource_package, resource_path) for loading file from .egg file