Where to put airflow_local_settings.py in Composer?

1.8k views Asked by At

In composer (airflow 1.10.10), is it possible to create an airflow_local_settings.py file? And if so where should it be stored? I need this as I need an initContainer for my pod.

Add a airflow_local_settings.py file to your $PYTHONPATH or to $AIRFLOW_HOME/config folder.

For me, the above statement is unclear for Cloud-composer, as this config folder in an env bucket would probably not be synced with a worker.

2

There are 2 answers

0
ejsan On

Based on Slack discussions in the Apache Airflow Community Slack. It is not supported yet.

0
Andrei Efimov On

airflow_local_settings.py can be provided into dags folder in Composer GCS bucket according to Airflow initialization implementation where DAGs folder is added to sys.path before the configuration folder ($AIRFLOW_HOME/config). Keep in mind that by doing so you are overriding the Composer default policies.