I have around 50 k xml files which I wanted to upload to ADLS/DBFS. Can this can be uploaded as a zip file and work on zip file from ADLS/DBFS. As of now its taking so much of time for uploading into ADLS/DBFS. I use dbfs cp -r which is not effective in this case. Also trying to upload through Databricks UI. Same case when trying to upload through storage explorer to ADLS Looking forward to valid suggestions:)

1

There are 1 answers

2
marat On

You can use AzCopy or Azure Storage Explorer to upload to ADLS. They use multiple upload threads and may end up being faster than dbfs cp -r which uploads files one by one.