Trying to run a Azure Data-factory pipeline containing a pig activity which runs on on-demand HDI cluster and reads from a azure-storage dataset linked service. The script reads input and store to a different similar dataset.
I am able to run the pipeline and the on-demand cluster also seems to be created but my activity failed with a pig error
ERROR 2118: Input path does not exist: wasb://mystorage.blob.core.windows.net/inp
"mystorage" is a general purpose storage account with standard tier. In this account i opened the blob link and created a container named "inp" which containes few uploaded files.
Any ideas what i am doing wrong. Thanks for any help.
Can you try using the
wasb[s]://<containername>@<accountname>.blob.core.windows.net/syntax?