It is difficult to find a good title.
I think the problem is widely familiar.
I have several UseCase folders, and in each there is a python script config.py.
Each config.py performs correctly iff run from a terminal in that very UseCase folder.
I'd like to launch each config.py automatically from yet another python script purge.py. And of course I want to do it in parallel.
The following is my attempt. It works when launch_config is called in a naive for-loop.
def launch_config(project):
os.chdir(project)
os.system('python config.py')
# build all projects in parallel
pool = multiprocessing.Pool(processes=multiprocessing.cpu_count())
#
for (filename,name) in targets:
#
project = os.path.join(usecase_dir,f"UseCase_{filename}")
#
pool.apply_async(launch_config, args=(project,) )
#
pool.close()
pool.join()
#
os.chdir(pwd)
When run in the pool, however, it just crashes from within python itself with a several-hundred-lines-long unintelligible error message. Are there any general mis-assumptions in my approach that can be identified?