Python Egg failing to load sub-modules in my project

Question

I am trying to create an egg package for my python project build using cement (builtoncement) library

My project structure looks like this

connector
    README.md
    requirements.txt
    setup.py
    __main.py__
    __init.py__
    lib
        __init__.py
        handlers
               __init__.py
                logging
                       __init__.py
                        LogStashHandler.py

The contents of the __main.py__ is

import sys
import os
from cement.core.foundation import CementApp
from lib.handlers.logging.LogStashHandler import LogStashHandler
……


class ConnectorApp(CementApp):


def main():
    log = None
    with ConnectorApp() as app:
        log = app.log



if __name__ == '__main__':
    main()

The contents of the setup.py is

#!/usr/bin/env python

import uuid
from setuptools import setup
import os
from pip.req import parse_requirements

def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname)).read()

BASE_DIR = os.path.dirname(os.path.realpath(__file__))
reqs_file = os.path.join(BASE_DIR, 'requirements.txt')
install_reqs = parse_requirements(reqs_file, session=uuid.uuid1())

setup(
    name="connectors",
    version="0.1",
    author="Pradeep Mishra",
    install_requires=["argparse", "requests", "pyyaml", "datetime", "cement"],
    packages=['lib'],
    long_description=read('README.md'),
    data_files=[ ('', ['__main__.py', ])],
    classifiers=[     
        "Programming Language :: Python :: 2.7"
    ],
)

I am installing the program using below commands

python setup.py build
python setup.py sdist
python setup.py bdist_egg

When I try to run the program by going into dist folder, I am getting following error:

(venv) Pradeeps-MacBook-Pro:dist pradeepmishra$ python connectors-0.1-py2.7.egg 
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "connectors-0.1-py2.7.egg/__main__.py", line 4, in <module>
ImportError: No module named handlers.logging.LogStashHandler

Please help. How can I tell the egg builder to locate the modules and sub-modules inside lib module

Answer 1

I found it myself. Instead of specifying the top level package in packages, I passed the function find_packages() and it worked like charm.

My setup function looks like this now #!/usr/bin/env python

import uuid
from setuptools import setup, find_packages
import os
from pip.req import parse_requirements


def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname)).read()

BASE_DIR = os.path.dirname(os.path.realpath(__file__))
reqs_file = os.path.join(BASE_DIR, 'requirements.txt')
install_reqs = parse_requirements(reqs_file, session=uuid.uuid1())

setup(
name="connectors",
version="0.1",
author="Pradeep Mishra",
install_requires=["argparse", "requests", "pyyaml", "datetime", "cement"],
packages=find_packages(),
long_description=read('README.md'),
data_files=[ ('', ['__main__.py', ])],
classifiers=[     
    "Programming Language :: Python :: 2.7"
],

)