How can I create external data source in BigQuery and connect to composer cloud sql?

200 views Asked by At

We have a composer environment in GCP and we want to export the airflow database in the tenant project to BigQuery. Is there a way to do so? Composer

1

There are 1 answers

3
guillaume blaquiere On

I searched in all the location of Composer (on GKE with the secret and the cloud SQL proxy), in the Airflow db connexion, I got all the required element to connect my external source to BigQuery and I got this error:

enter image description here

So, you can't.....


EDIT 1:

Firstly, you can use the automatically configured connexion

Then, use the MySqlOperator to request the database and do what you want. Sometime it's easier to use the MySqlHook.

enter image description here