How to access the Hydra config object at runtime

Question

I need to change the output/working directory of the hydra config framework in such a way that it lies outside of my project directory. According to my understanding and the doc, config.yaml would need to look like this:

exp_nr: 0.0.0.0
condition: something
hydra:
  run:
    dir: /absolute/path/to/folder/${exp_nr}/${condition}/

In my code, I then tried to access and set the path like this:

import os
import hydra
from omegaconf import DictConfig


@hydra.main(config_path="../../config", config_name="config", version_base="1.3")
def main(cfg: DictConfig):
    print(cfg)
    cwd = os.getcwd()
    print(f"The current working directory is {cwd}")
    owd = hydra.utils.get_original_cwd()
    print(f"The Hydra original working directory is {owd}")
    work_dir = cfg.hydra.run.dir
    print(f"The work directory should be {work_dir}")

But I get the following output and error:

{'exp_nr': '0.0.0.0', 'condition': 'something'}
The current working directory is /project/path/subdir/subsubdir
The Hydra original working directory is /project/path/subdir/subsubdir
Error executing job with overrides: ['exp_nr=1.0.0.0', 'condition=somethingelse']
Traceback (most recent call last):
  File "/project/path/subdir/subsubdir/model.py", line 13, in main
    work_dir = cfg.hydra.run.dir
omegaconf.errors.ConfigAttributeError: Key 'hydra' is not in struct
    full_key: hydra
    object_type=dict

I see that hydra.run.dir doesn't appear in the cfg dict printed first but how can I access the path through the config if os.getcwd() isn't set already? Or what did I do wrong?

The path is correct as I already saved files to the folder before integrating hydra and if the process isn't killed due to the error the folder also gets created but hydra doesn't save any files to it, not even the log file with the parameters it should save by default. I also tried to set the path relative to the standard output path or having an extra config parameter work_dir: ${hydra.run.dir} (returns an Interpolation error).

Answer 1

You can access the Hydra config via the HydraConfig singleton documented here.

from hydra.core.hydra_config import HydraConfig

@hydra.main()
def my_app(cfg: DictConfig) -> None:
    print(HydraConfig.get().job.name)