luigi running external program can't locate bash file

1.1k views Asked by At

I am 'simply' trying to solve how to run an external program from luigi. Utilising the ExternalProgramTask from luigi.contrib.external_program.

The luigi class is:

from luigi.contrib.external_program import ExternalProgramTask
import luigi

class PoincarreEmbedding(ExternalProgramTask):
    file_path = 'examples/tmp/3'

    def program_args(self):
        return [". embeddings.sh"]

Luigi runs and fails as it cannot find the file.

INFO: Running command: . embeddings.sh
ERROR: [pid 43886] Worker Worker(salt=404030211, workers=1, 
host=Roberts-MacBook-Pro.local, username=rhsmith, pid=43886) failed    
PoincarreEmbedding()
Traceback (most recent call last):
File "/Users/rhsmith/luigi/luigi/worker.py", line 194, in run
new_deps = self._run_get_new_deps()
File "/Users/rhsmith/luigi/luigi/worker.py", line 131, in _run_get_new_deps
task_gen = self.task.run()
File "/Users/rhsmith/luigi/luigi/contrib/external_program.py", line 98, in run
stderr=tmp_stderr
File "/Users/rhsmith/anaconda2/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/Users/rhsmith/anaconda2/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

The '. embedding.sh' file is located in the same directory as where the module/task is run from.

Have been studying this thread but to no avail. I am sure it simple :/

0

There are 0 answers