.Conf file manipulation from multiple python scripts

Question

I'm using the ConfigParser module to read/write a .conf file from two Python scripts. One script is only reading data, while the other may also write to the .conf file. Considering this context is possible to end with corrupt data ? Or the ConfigParser is preventing such situations ?

Cheers

Answer 1

ConfigParser itself doesn't know how to open and write physical files and therefore cannot prevent races. You pass a file-like object to write() and it's up to you to make sure that files you're changing are updated atomically. On POSIX systems like Linux, this is typically done by writing to a temporary file and renaming it to the final name when the writing has completed.

An atomic rename requires that the source and destination file are on the same filesystem, and an easy way of guaranteeing this is making sure that the files are in the same directory.

import ConfigParser
import os
import tempfile

FILENAME = '/some/path/test.config'

config = ConfigParser.SafeConfigParser()
config.read([FILENAME])
config.add_section('foo')
with tempfile.NamedTemporaryFile(dir=os.path.dirname(FILENAME),
                                 delete=False) as tempfile:
  config.write(tempfile)
os.rename(tempfile.name, FILENAME)

This assumes that you only have one concurrent writer of the config file in addition to the one or more concurrent readers.

Doing an atomic replacement on Windows is less trivial. See e.g. Is an atomic file rename (with overwrite) possible on Windows?. Also relevant to the discussion is How to safely write to a file? – depending on the file system implementation (and its mount options) it's possible that the metadata change (the rename) completes before the data is persisted to the file.