Import module from directory one level up when running script in subdirectory

Question

Say I have a collection of scripts organized for convenience in a directory structure like so:

root
│   helpers.py
│
├───dir_a
│       data.txt
│       foo.py
│
└───dir_b
        data.txt
        bar.py

foo.py and bar.py are scripts doing different things, but they both load their distinct data.txt the same way. To avoid repeating myself, I wish to put a load_data() function in helpers.py that can be imported while running scripts in subdirectories dir_a and dir_b.

I have tried to import helpers in dir_a/foo.py using a relative import like this:

from ..helpers import load_data

foo_data = load_data('data.txt')

# ...rest of code...

But directly running foo.py as a script fails on line 1 with an ImportError:

Traceback (most recent call last):
  File "path/to/file/foo.py", line 1, in <module>
    from ..helpers import load_data
ImportError: attempted relative import with no known parent package

Is the spirit of what I'm attempting possible in python (i.e. simple and without modifying sys.path)? From Case 3: Importing from Parent Directory on this site, the answer seems to be no, and I need to restructure my collection of scripts to avoid importing one level up.

Bonus question: if not possible, what is the rationale for why not?

Answer 1

If you create an empty __init__.py in the root directory (which turns root into a package) and then call your foo.py script like this:

python -m root.dir_a.foo

it should work, I think, so long as you call it from above 'root'. I've just tried something similar on my machine and it worked.

Alternatively, because you have now made 'root' a package, you can just:

from root import helpers

Of course, I assume your root directory isn't really called 'root', in which case you should change it appropriately. I'm also assuming you're using Python3 - it won't work in 2.7.

As to the rationale of why this isn't possible without making root a package, I think the idea is that your code should be location agnostic. In other words, you shouldn't rely on your foo.py knowing where it is on the system. I'm not an expert, but actually it maybe that foo.py doesn't know this, in which case it has no way of resolving ../helpers.

Answer 2

The correct way to run a python script is from the parent directory of the root package as suggested in the answer by @JoeTodd. If you run the script directly from its location then it has no way of knowing it is in a package hence the error (no known parent package).

Now as to the data.txt, python when given a relative path it has to resolve it to an absolute path - it can't open 'data.txt' it needs to open c:/path/to/data.txt. Relative paths are therefore concatenated to the current working directory which is by default the directory the script is run from (which is also added to the sys.path and hence the packages inside it are resolved). So when you run from the parent dir of root, python looks for c:/.../parent dir of root/data.txt which doesn't exist. The correct solution is to not use relative paths - so:

path_to_data = os.path.join(os.path.dirname(__file__), 'data.txt'))
foo_data = load_data(path_to_data)

Answer 3

Add the following lines to the top of your foo.py script,

import sys
sys.path.append('../root')

Which will set the path to a module a level higher. You will then be able to do your import in the foo.py script as follows,

from helpers import load_data

I was stuck on a very similar issue and found that this is the simplest way of resolving this issue