Cannot pickle a Python Class instance

Question

Here I have this class definition class definition. When I run below code, it raises following errors.

sm = SaliencyMaskSlic()
operations = [('img_resize', img_resize), ('sal_mask', sm.transform)]
args_list = [{'h_size':258}, {'cropped':True}]

pre_pipeline = Pipeline(ops=operations, arg_list=args_list)
ch = ColorHist('RGB', [6,6,6], [2,2], center=True, pre_pipeline = pre_pipeline)

dill.dump(ch, open('erogol.pkl','wb'))
...
dill.loads('erogol.pkl')


---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-11-c8a5937780b5> in <module>()
----> 1 dill.loads('erogol.pkl')

/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in loads(str)
    158     """unpickle an object from a string"""
    159     file = StringIO(str)
--> 160     return load(file)
    161 
    162 # def dumpzs(obj, protocol=None):

/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in load(file)
    148     pik = Unpickler(file)
    149     pik._main_module = _main_module
--> 150     obj = pik.load()
    151     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    152         try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)

/usr/lib/python2.7/pickle.pyc in load(self)
    856             while 1:
    857                 key = read(1)
--> 858                 dispatch[key](self)
    859         except _Stop, stopinst:
    860             return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_appends(self)
   1185     def load_appends(self):
   1186         stack = self.stack
-> 1187         mark = self.marker()
   1188         list = stack[mark - 1]
   1189         list.extend(stack[mark + 1:])

/usr/lib/python2.7/pickle.pyc in marker(self)
    872         mark = self.mark
    873         k = len(stack)-1
--> 874         while stack[k] is not mark: k = k-1
    875         return k
    876 

IndexError: list index out of range

Basically I have one class instance using another class instance inside. I also used cPickle but it raises as I dump;

TypeError: can't pickle instancemethod objects

Any idea for the solution ?

Answer 1

This isn't a pickling error. You can't pickle class instances with pickle or cPickle, but you can with dill. Your code has a bug somewhere that's giving you an IndexError.

Also better than your class having a dump and load method, you might just use dump and load from dill directly... then if you are doing something complicated, you can still add a __getstate__ and __setstate__ method.

Also, your loading from a pickled file, has a bug. You are doing this:

    self = dill.loads(in_path)

While you should (1) be using dill.load instead, and (2) load to _self, and then replace the relevant state.

    _self = dill.load(in_path)
    self.nbins = _self.nbins
    self.mask = _self.mask
    # and so on... (or update all at once using `__dict__`)