When automatically importing modules from a subfolder, their imports fail

Question

I've read through a couple of similar questions, notably this one about imp.load_module which seems to be close to what I want, but I can't understand why I'm still getting ImportErrors. Here is my folder hierarchy:

program\
  __init__.py
  main.py
  thirdparty\
    __init__.py
    css\
      __init__.py
      css.py
      utils\
        __init__.py
        http.py

In main.py I have the following code. This is intended to search the thirdparty\ directory and load each module it finds. Each module is in its own separate directory.

import os
import imp

for root, dirs, files in os.walk("thirdparty"):
    for source in (s for s in files if s.endswith(".py")):
        name = os.path.splitext(os.path.basename(source))[0]
        m = imp.load_module(name, *imp.find_module(name, [root]))

The problem is that css.py happens to use its own subfolder that it loads stuff off of, utils. It has a line in it that says:

from utils import http

And that is where it fails. I get this error when I run main.py.

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    m = imp.load_module(name, *imp.find_module(name, [root]))
  File "thirdparty/css/css.py", line 1, in <module>
    from utils import http
ImportError: No module named utils

I'm stumped. css.py is self contained in its own folder, and when I run css.py separately it imports utils just fine. What is causing this?

Answer 1

Maybe you can solve this by changing the import to:

from .utils import http

Or by adding the folder you import to the Python Path:

sys.path.append(os.path.join(root, source))

When you import modules in thirdparty, the place Python looks for modules is still the main directory. The initial import works, because you give the right path to imp.find_module, but after that Python has no idea where to look for the modules.