I'm using the official Airflow Helm to try out the new LocalKubernetesExecutor feature. However, I also wanted the following:
- allow helm to bring up the postgresql container
- change the credentials, username and database for that container.
I don't really know how to do this. I tried overriding the file section postgresql:
postgresql:
enabled: true
postgresqlPassword: airflow
postgresqlUsername: airflow
But it doesn't seem to do anything. I fetched the connection that was created and decoded the base64 string:
kubectl get secret airflow-airflow-metadata -o json
{
"apiVersion": "v1",
"data": {
"connection": "cG9zdGdyZXNxbDovL3Bvc3RncmVzOnBvc3RncmVzQGFpcmZsb3ctcG9zdGdyZXNxbC5haXJmbG93OjU0MzIvcG9zdGdyZXM/c3NsbW9kZT1kaXNhYmxl"
},
"kind": "Secret",
"metadata": {
"annotations": {
"meta.helm.sh/release-name": "airflow",
"meta.helm.sh/release-namespace": "airflow"
},
"creationTimestamp": "2022-11-20T20:14:30Z",
"labels": {
"app.kubernetes.io/managed-by": "Helm",
"chart": "airflow",
"heritage": "Helm",
"release": "airflow",
"tier": "airflow"
},
"name": "airflow-airflow-metadata",
"namespace": "airflow",
"resourceVersion": "7643",
"uid": "14fff962-aec8-4862-b598-4ae3dbeca26f"
},
"type": "Opaque"
}
When I decode the connection:
echo cG9zdGdyZXNxbDovL3Bvc3RncmVzOnBvc3RncmVzQGFpcmZsb3ctcG9zdGdyZXNxbC5haXJmbG93OjU0MzIvcG9zdGdyZXM/c3NsbW9kZT1kaXNhYmxl | base64 -D
postgresql://postgres:[email protected]:5432/postgres?sslmode=disable%
I'm beginning to think that it is something very simple I'm missing, or it can't be done.
So, the question is - how to override postgres user, password, and database when using official Airflow Helm? Can it be done?
connections in airflow cannot be overridden. you have to delete the connection and import the updated connection json.