Does GCP Dataform provide a mechanism to detect the completion of external upstream source data tasks? For instance, in Airflow, I can check if the source table is ready before executing a DAG. Is there a similar capability in Dataform to verify if the source data is ready? If not, should I rely on Airflow to handle such orchestration and schedule Dataform workflow to run within Airflow?
I searched on Google's documentation and community. I am looking for an answer for my organization.
After consulting Google Cloud community, I learned that I need to orchestrate it using Airflow to handle external source dependencies.