pkg_resources.DistributionNotFound: The 'fairseq' distribution was not found

41 views Asked by At

I'm trying to reproduce the work of a paper that uses fairseq code in a Singularity container. In the Singularity recipe I install fairseq code with pip install --editable . and it appears to work well:

# singularity build log [...]
Installing collected packages: cython, fairseq
  Running setup.py develop for fairseq
Successfully installed cython-3.0.4 fairseq
# [...]

But then I try to execute a fairseq-preprocess command and I get the following error:

Singularity> fairseq-preprocess --source-lang pt --target-lang pt --trainpref data-bin/sp25000/train --destdir data-bin/sp25000/ --dataset-impl raw
Traceback (most recent call last):
  File "/usr/local/bin/fairseq-preprocess", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3254, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3237, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3266, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 584, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 901, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'fairseq' distribution was not found and is required by the application

I have done this exact same procedure before in a different computer and it worked, so I'm not sure what I did wrong. I believe the problem may be that the installation is done in an unconventional location. How can I know where it is being installed? There are other issues that can explain this?

If it helps, I'm trying to reproduce this code.

EDIT:

I tried including the path to fairseq in $PYTHONPATH but got the same error.

0

There are 0 answers