How to pass a file as hyperparameters for argparse

Question

I am trying to reproduce a pipeline from github. Since the Idea is to reproduce it, I don't want to change the code. Onde of the scripts ask for hyperparameters to be passed as a dictionary, as in -p {'param1' : 'p1', 'param1' : 'p2' ...} and so forth. These parameters are processed with argparse.

I have a long list of parameters. Is it possible to pass the dictionary of hyperparameters as a file?

Thanks!

Answer 1

If you don't want to modify the original script at all, you can create a wrapper script that reads the parameters from a JSON (let's call it params.json) and then calls the original script with these parameters.

{
  "param1": "p1",
  "param2": "p2"
}

and pass it using:

params=$(cat params.json)
python your_script.py -p "$params"

Answer 2

argparse itself, by default, treats an argument starting with @ as the name of a file from which it should read further arguments, one per line. So if you create a file named args.txt containing

-p
{'param1' : 'p1', 'param1' : 'p2' ...}

(assuming the "dictionary" is really supposed to be passed as a single argument) then

script.py @args.txt

should be equivalent to

script.py -p "{'param1': 'p1', 'param2': 'p2'}"