U-SQL Error while using REFERENCE ASSEMBLY

Question

I created a U-SQL library using Azure APIs and register the assembly on Azure cloud with all dependencies. I added this library with my U-SQL project and added below line On my U-SQL script

USE master;
REFERENCE ASSEMBLY [AzureLibrary];

At the time of using functions or methods I created in library, I got below error message.

Inner exception from user expression: Could not load file or assembly 'Microsoft.Azure.Management.DataLake.Store, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. Current row dump: result: "zzz"

Please help, I tried several ways but the same message appears every time.

======================================

I am using below code

public static int DelTest()
        {
            int Retval = 1;
            try
            {
                var secretKey = <secret key>;
                var appId = <app id>;
                var tenantId = <tenant id>;
                string _adlsAccountName = <adls account name>;
                DataLakeStoreFileSystemManagementClient _adlsFileSystemClient;

                var context = new AuthenticationContext("https://login.windows.net/" + tenantId);
                ClientCredential clientCredential = new ClientCredential(appId, secretKey);
                var tokenResponse = context.AcquireTokenAsync("https://management.azure.com/", clientCredential).Result;
                var accessToken = tokenResponse.AccessToken;
                using (var client = new HttpClient())
                {
                    client.DefaultRequestHeaders.Add("Authorization", "Bearer " + accessToken);
                    client.BaseAddress = new Uri("https://management.azure.com/");
                    // Now we can party with our HttpClient!
                }

                ServiceClientCredentials creds = new TokenCredentials(tokenResponse.AccessToken);

                _adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
                DataLakeStoreAccountManagementClient _adlsClient = new DataLakeStoreAccountManagementClient(creds);
                _adlsFileSystemClient.FileSystem.Delete(_adlsAccountName, "/raw_files/sls_tty_typ.txt");
            }
            catch (Exception)
            {
                Retval = 0;
                throw;
            }
            return Retval;
        }

I register it using "Register Assembly"

I used below code in U-SQL script

USE [master];
REFERENCE ASSEMBLY [USQLCSharpProject1];

@ext =
    EXTRACT result string
    FROM "/raw_files/test_file.txt"
    USING Extractors.Csv();

@ext1 =
    SELECT result,
           USQLCSharpProject1.Class1.DelTest() AS del_success FROM @ext;

OUTPUT @ext1
TO "/raw_files/test_file1.txt"
USING Outputters.Csv();

During execution, I got below error message.

Please help me to resolve this issue.

Please find packages.config file

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Azure.Management.DataLake.Analytics" version="1.0.1-preview" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.Store" version="1.0.2-preview" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.StoreUploader" version="1.0.0-preview" targetFramework="net452" />
  <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.13.8" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.2" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.2" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="6.0.8" targetFramework="net452" />
</packages>

Answer 1

For this issue, Please check below scenarios:

1) Make sure the assembly's property Copy Local is true:

2) When Register Assembly, please make sure the Managed Dependencies are selected:

3) please check bin/Debug folder to make sure this assembly is existed. If the register assembies larger than 15kb, please make sure it is existed in Data Lake store

Answer 2

can you please forward me a repro project and the exact steps you take to usql (at) Microsoft?

Looking at your code above, I have one caveat: I see you try to do a direct HTTP call. The U-SQL vertex mode in ADLA does not allow you to make HTTP calls to avoid accidental DDOS incidents. So even if the registration would work, I am doubtful your code would actually behave as you expect.

What are you trying to achieve?