I am 'simply' trying to solve how to run an external program from luigi. Utilising the ExternalProgramTask from luigi.contrib.external_program.
The luigi class is:
from luigi.contrib.external_program import ExternalProgramTask
import luigi
class PoincarreEmbedding(ExternalProgramTask):
file_path = 'examples/tmp/3'
def program_args(self):
return [". embeddings.sh"]
Luigi runs and fails as it cannot find the file.
INFO: Running command: . embeddings.sh
ERROR: [pid 43886] Worker Worker(salt=404030211, workers=1,
host=Roberts-MacBook-Pro.local, username=rhsmith, pid=43886) failed
PoincarreEmbedding()
Traceback (most recent call last):
File "/Users/rhsmith/luigi/luigi/worker.py", line 194, in run
new_deps = self._run_get_new_deps()
File "/Users/rhsmith/luigi/luigi/worker.py", line 131, in _run_get_new_deps
task_gen = self.task.run()
File "/Users/rhsmith/luigi/luigi/contrib/external_program.py", line 98, in run
stderr=tmp_stderr
File "/Users/rhsmith/anaconda2/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/Users/rhsmith/anaconda2/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
The '. embedding.sh' file is located in the same directory as where the module/task is run from.
Have been studying this thread but to no avail. I am sure it simple :/