Looping over multiple data files in Python 2.7

202 views Asked by At

I'm having difficulty reading multiple data files in a loop. I would like to solve my mathematical model with very many different instances. I have each instance stored in a txt file. An instance is defined by some parameters (numbers, arrays and matrices). For example, instance InstanciaSEC_E0_m1_J3_K2.txt contains the following data:

E = 0
m = 1
J = 3
K = 2
p = {0: 0.696969696969697, 1: 0.30303030303030304}
UDC = {(0, 1): 9.0, (1, 2): 10.0, (0, 0): 5.0, (1, 1): 6.0, (1, 0): 9.0, (0, 2): 6.0}
UDU = {(0, 1): 5.0, (1, 2): 4.0, (0, 0): 2.0, (1, 1): 4.0, (1, 0): 1.0, (0, 2): 3.0}
UAC = {(0, 1): 1.0, (1, 2): 0.0, (0, 0): 2.0, (1, 1): 3.0, (1, 0): 4.0, (0, 2): 0.0}
UAU = {(0, 1): 9.0, (1, 2): 10.0, (0, 0): 5.0, (1, 1): 6.0, (1, 0): 10.0, (0, 2): 6.0}

To read a single instance, I can simply change it into a .py file and import it:

import InstanciaSEC_E0_m1_J3_K2 as I

and then simply access the information within by doing:

 I.E=E
 I.m=m
 I.UDC=UDC

and so on. Once I have the instance information, I can construct my model and solve it and retrieve the necessary information about how the solving went.

I would like to be able to do the same for many such instance files. Ideally, I should be able to import the data files iteratively, extract the information, solve my model, and move on to the next data file. I don't know if one can do imports in a loop within a main function and get away with it...so I tried the following based on the answer given in https://stackoverflow.com/a/924719 but it's not working...I define the following function to retrieve the data from the instance file:

def getVarFromFile(filename):
    import imp
    f = open(filename)
    global data
    data = imp.load_source('data', "", f)
    f.close()

Then, inside the main code I do the following:

 Exp = 5
 for K in range(50,101,50):
     for J in range(5,6,1):
         for m in range(2,3,1):
             for E in range(Exp):
                getVarFromFile('/Users/Carlos/Desktop/Stab2/InstanciasSeguridad/InstanciaSEC_E%s_m%s_J%s_K%s.py' %(E,m,J,K))
                print data.E
                print data.m
                print data.J
                print data.K (etc.)
                print "/Users/Carlos/Desktop/Stab2/InstanciasSeguridad/InstanciaSEC_E%s_m%s_J%s_K%s.py" %(E,m,J,K)

The last print confirms that I am indeed looping through the data files but the info I'm retrieving is not correct..To my eyes the only shady bit is the imp.load_source() function as what it does is not immediately clear to me (even after reading the documentation). Any help would be great. Maybe I'm going about all this the wrong way and there is a much simpler way of looping over and retrieving variables from data files.

0

There are 0 answers