cannot import my util module

Question

I'm using sklearn.externals.joblib to persist a classifier model to the disk which in reality uses pickle module at lower level.

I create a custom CountVectorizer class named StemmedCountVectorizer and saved it in util.py, then used it in the script for persisting the model

import util

from sklearn.externals import joblib

vect = util.StemmedCountVectorizer(stop_words='english', ngram_range=(1,1))

bow = vect.fit_transform(sentences)

joblib.dump(vect, 'vect.pkl') 

This my project structure using Flask:

   |- sentiment/
     |- run.py
     |- my_app/
       |- analytic/
         |- views.py
         |- util. py
         |- vect.pkl

I run the app with python run.py and try to load the persisted object with joblib.load in views.py but it does not work, I imported the util module but I receive the error:

ImportError: No module named util

can anybody give a solution to this? thanks

Answer 1

Looks like a package/pythonpath problem. The system need to know where to locale your modules. Do you have __init.py__ in my_app and analytic folder? The __init__.py file mark directories on disk as Python package directories. And the structure should be like this

   |- sentiment/
     |- run.py
     |- my_app/
       |- __init__.py
       |- analytic/
         |- __init__.py
         |- views.py
         |- util. py
         |- vect.pkl

then in your run.py, try import with

import my_app.analytic.utils

or

from my_app.analytic.utils import <yourClassName>

for details of python package, check here. And be aware of namespace problem.