How to connect and read/write to Azure ADLS Gen2 or Blob Store using Shared Access Signature with Databricks

133 views Asked by At

I created SAS keys for my Azure Storage account, see image below.

Can someone show me how to connect to the Storage account with Databricks using the one of the Keys listed below:

enter image description here

I tried the suggested answer, however I'm getting the following error

[enter image description here

Any thoughts

enter image description here

The following shows the SAS token I used, but still getting the error enter image description here

1

There are 1 answers

4
JayashankarGS On BEST ANSWER

Copy the SAS token and use the below code.

spark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net", "SAS")
spark.conf.set("fs.azure.sas.token.provider.type.<storage-account>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider")
spark.conf.set("fs.azure.sas.fixed.token.<storage-account>.dfs.core.windows.net", "?sv=...SAS_token")

spark.read.csv('abfss://[email protected]/outfromVS.csv').display()

Output:

_c0 _c1
0 2016-07-28,CA,1,A,A1,7.92
1 2016-07-29,CA,1,A,A1,7.88
2 2016-07-30,CA,1,A,A1,8.0
3 2016-07-31,CA,1,A,A1,7.94
4 2016-08-01,CA,1,A,A1,8.0
5 2016-08-02,CA,1,A,A1,7.98
6 2016-08-03,CA,1,A,A1,7.99

Refer this for more information.