How can we write data to Azure synapse dedicated sql pool from Azure Databricks using a Service principal?

Question

I am trying to use below code to write the data to synapse dedicated sql pool table.

The Data is stored in ADLS Gen2 and I am trying to write a dataframe into a sql table

I also have a service principal created for Azure Databricks that I am also using in Synapse as db_owner,

While running the code - I get below error:

java.lang.IllegalArgumentException: Could not retrieve Credential Passthrough token. Please check if Credential Passthrough is enabled

Can some please help explain what is wrong here because my cluster shows passthrough enabled

Answer 1

I got same error when I tried in my environment.

You can see here, I have not enabled the credential passthrough for the cluster and still got the error.

To resolve this error, make sure you have the correct permissions as mentioned in this documentation.

And also, you can try with two service principals one for the storage account another for the synapse workspace. Add these in the databricks ini script as mentioned in this documentation.