Python 2.5 - Update folder of text files from values stored in a lookup list

Question

I am trying to write a script that will update a folder of text files based on a lookup table. The lookup table is filename, oldpath, new path. The script looks in each text file for the filename, if there it updates the oldpath in the same line with the new path. The code is:

# Import
from array import *
import glob

# Specify the lookup table, to keep it simple drop it in with the workspaces
Lookup = "./Lookup.csv"

# Specify the 
Workspaces = glob.glob('./*.wor')

# Open the Lookup table
for line in open(Lookup).readlines():
    # Create the list to store the lookup parameters of the lookup line
    LookupList = []
    # Split the lookup csv at the comma
    for i in line.split(","):
      #print i
        LookupList.append(i)
# Use the list parameters to populate variables (could use list parameters but
# easier for now to assign to variable)
FileName = LookupList[0]
OldPath = LookupList[1]
NewPath = LookupList[2]

# We now have variables to use in the replace statement
# Use the Workspaces Glob to loop through the workspaces
for wor in Workspaces:
    # Try to open the the first workspace (text file)
    f = open(wor, 'r+')
    # Loop through the open file 
    for line in f.readlines():
        # For each line check whether the current list value (FileName) is in the line
        if '"' + OldPath + '"' in line:
            print line
            # Update the line, replacing the old path with the new path.
            line.replace(OldPath, NewPath);
    # Close the workspace file        
    f.close()

It all seems to work as it should, the print statement 5 lines from the end has found the correct lines that contain the search strings from the lookup, but the file is not updated.

I have read as much as I can find on file opening modes and updating files but there is no obvious solution. I guess the issue is reading/writing to the same file. The route I have chosen is open the lookup and embed the files to be changed loop. The alternative is open the file and then loop through the lookup.

Happy to write the updated file out to another name/folder, the issue there is that if you loop through the files to update, update the row based on the lookup, when you get to the next row of the lookup it will overwrite the previous lookup change.

Any ideas gratefully received. Aplogies if the description seems convoluted, happy to clarify any areas where the aim is not obvious.

Thanks

Paul

Answer 1

f.readines()

returns a list of strings, and you are iterating over these strings. So, when you make an edit on that string using

line.replace(...)

you are not changing the text file. Rather, you are changing the string that you have read in.

Your approach should be to write each line to a temp list, then write that temp list to a file, like:

f = open(wor, 'r+')
new_lines = []
for line in f.readlines():

    if '"' + OldPath + '"' in line :
        line.replace(OldPath, NewPath);
    new_lines.append(line)

f.close()

file("path/to/your/new/or/temp/file","w")
file.write("\n".join(new_lines))
file.close()

Answer 2

There are issues with the way that you are appending the LookupList, as you repeatedly assign it to [], which makes it empty again, so only the last iteration is kept. But that aside, this code for writing should do what you are intending it to:

# We now have variables to use in the replace statement
# Use the Workspaces Glob to loop through the workspaces
for wor in Workspaces:
    # Handles opening and closing of input and output files
    with open(wor, 'r'),open("new_" + wor,'w') as infile,outfile:
        # Loop through the input file
        for line in infile.readlines():
            # For each line check whether the current list value (FileName) is in the line
            if '"' + OldPath + '"' in line:
                print line
                # Update the line, replacing the old path with the new path.
                line.replace(OldPath, NewPath);
            outfile.write(line)

Answer 3

This is the code I have used which has worked for testing. It contains a large slice of Khol's suggestion so I have credited him with the answer. I am quite new to Python so the code may be a 5 yard beauty but it produces the results we are looking for.

# Import
from array import *
import glob

# Specify the lookup table, to keep it simple drop it in with the workspaces
Lookup = "./Lookup.csv"

# Specify the list of workspaces (text files to update)
Workspaces = glob.glob('./*.wor')

# Open the Lookup table
for line in open(Lookup).readlines():
    # Create the list to store the lookup parameters of the lookup line
    LookupList = []
    # Split the lookup csv at the comma
    for i in line.split(","):
      # Add items to the lookup list
        LookupList.append(i)

    # Assign the list value to a variable (could skip assigning to variable),
    # strip CR from value or it will add it to the output string
    FileName = LookupList[0].strip()
    NewPath = LookupList[1].strip()

    # Loop through the workspaces
    for wor in Workspaces:

        # Open the workspace
        f = open(wor, 'r+')

        # Create list 
        WorList = []

        # Read through the workspace and use it to populate the list
        for line in f.readlines():
            WorList.append(line)
        f.close()

        # Loop through the workspace list looking for the FileName string (from the lookup list)
        for position, item in enumerate(WorList):
            if " " +  FileName + " " in item:
                # If the filename is found then create a string with the new file name in the old row structure
                Newval = "Open Table " + '"'+ NewPath + '" ' + "As " + FileName + " Interactive\n"
                # Update the value in the list based on the list position
                WorList[position] = Newval;

        # Open the output file (this is the same as the original input file)
        file=open(wor,"w")

        # Work through the list and write it to the file
        for s in WorList:
            file.write(s)
        file.close()