How to update the Data Catalog tag values through airflow CloudDataCatalog operators

Question

I would like to update Data Catalog tags through airflow CloudDataCatalog operators and schedule the DAG to update the tag values daily. This is to capture the ETL status on a BigQuery table and feed that status to Data Catalog tag. Could you please help me on what approach I should take?

I explored and found airflow offers Data Catalog related operators but I could find any actual use to alter the tag values in any platform.

Answer 1

I have not used these operators before but sounds like the CloudDataCatalogUpdateTagOperator should be exactly what you need.

I found this example code in the provider docs:

update_tag = CloudDataCatalogUpdateTagOperator(
    task_id="update_tag",
    tag={"fields": {FIELD_NAME_1: TagField(string_value="new-value-string")}},
    update_mask={"paths": ["fields"]},
    location=LOCATION,
    entry_group=ENTRY_GROUP_ID,
    entry=ENTRY_ID,
    tag_id=tag_id,
    gcp_conn_id=YOUR GCP AIRFLOW CONNECTION # added this line in case you don't use the default conn id string
)