How to create configuration file in Python where need to have os.path.join?

Question

I have a couple of Python modules, something like:

prepare_data.py process_data.py generate_report.py

I take an Excel file, process it and then create a .pdf report. It's very linear, procedural programming. In a main caller.py module, I call the functions one after another:

prepare_data.do1() prepare_data.do2() generate_report.do3()

etc.

I also have a Configuration.py module where I store global variables and settings that are imported in prepare_data.py, process_data.py, and generate_report.py.

However, along the execution, I am creating multiple intermediate files in the same folder where the Excel file is stored. In Configuration.py, it is very handy to have multiple variables that are created based on the location of the source Excel file:

excel_file = r"C:\Data\InData.xlsx"
scratch_folder = os.path.dirname(excel_file)
out_pdf = os.path.join(scratch_folder,'PdfReport.pdf')

#scratch_folder is C:\Data

Then in the modules that actually do the work, I would like to import the Configuration.py and then use the variables from this module:

... import Configuration as cfg create_pdf(cfg.out_pdf) ...

However, as I call caller.py and supply as an argument the path to Excel file, I cannot store the path to Excel file in Configuration.py.

If I use solutions like ConfigParser or just plain .conf file, if I understand it right, I cannot take advantage of os.path.join() and other Python functions for constructing paths.

What would be an efficient way to organize the configurations in my particular case?

Answer 1

Could you create a class?

in Configuration.py you would just declare a variable:

foo = None

In your main program you would declare a class and as soon as you have excel_file path available, create an instance:

class ExcelFileClass(object):
    def __init__(self, ef):
        self.scratch_folder = os.path.dirname(ef)
        ...

Configuration.foo = ExcelFileClass(excel_file)

In every module importing Configuration you would be able to access Configuration.foo.scratch_folder and other class variables. You could also add a get method if you want to do validation, error handling or something else to your class variables.

Hannu