joblib: cannot load dumped joblib file from another directory

3.5k views Asked by At

I am trying to load a dumped joblib file from another directory, it's giving the following error

Traceback (most recent call last):
  File "D:\dir\loader.py", line 5, in <module>
    foob = joblib.load(r'D:\foob.joblib')
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\site-packages\joblib\numpy_pickle.py", line 585, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\site-packages\joblib\numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1088, in load
    dispatch[key[0]](self)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1376, in load_global
    klass = self.find_class(module, name)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1426, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'foo'  

Minimal code to reproduce the error
The directory looks like this

D-
   \
    foo.py
    dumper.py
    foob.joblib
    dir-
        \
         loader.py 

foo.py

class Foo:
    def __init__(self):
        self.s = "here at Foo"
    def __repr__(self):
        return "fooObject"

dumper.py

import joblib
import foo

foob = foo.Foo()
joblib.dump(foob,'foob.joblib')

loader.py

import joblib
foob = joblib.load(r'D:\foob.joblib')
print(foob)  

If I try to load the foob.joblib from the same directory(D) it works fine, the error occurs while running loader.py from different directory

1

There are 1 answers

2
fnokke On BEST ANSWER

joblib.load() requires any modules required for unpickling to be in the path. So you need to add the location of foo.py the system path at loading time somehow.

For instance add:

import sys
import os
this_dir = os.path.dirname(__file__) # Path to loader.py
sys.path.append(os.path.join(this_dir, <rel_path_to_foo.py>))

to the top of loader.py