I am trying to use Azure Data Factory managed Airflow for my use cases. I am trying to use Azure Purview as data governance tool and want to use airflow to emit lineage to it.
I am trying to follow the Microsoft Document here. However, what i am stuck at is after i create event hub how do i configure the .yml file as Azure does it for me and i have no access to it. I am also not able to create the OPENLINEAGE_URL correctly pointing to Eventhub and it is throwing me 401(assuming i am not providing SAS correctly). This is the URL i am providing:
https://{event-hub-namespace}.servicebus.windows.net/{event-hub}/messages
Azure Data Factory Managed Airflow integration with Purview to get Lineage
122 views Asked by Nidutt At
1
i am also on the same process but have not reached the expected result yet.
Regarding the configuration of the .yml file there is a suggestion at the link below: https://learn.microsoft.com/en-us/answers/questions/1405026/azure-data-factory-managed-airflow-integration-wit
mentioning the following:
Did you try to create a new Variable and provide the YML contents?