In Azure, I'm setting up an Azure Function with a blobtrigger, it uses a SAS connectionstring to a blobstorage account in the same subscription, in the same resource group. When starting the function, in either the logstream in kudu or app insights, I get the following exception:
Azure.RequestFailedException: at Azure.Storage.Queues.QueueRestClient+<GetPropertiesAsync>d__14.MoveNext (Azure.Storage.Queues, Version=12.14.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Azure.Storage.Queues.QueueClient+<GetPropertiesInternal>d__61.MoveNext (Azure.Storage.Queues, Version=12.14.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Azure.Storage.Queues.QueueClient+<ExistsInternal>d__52.MoveNext (Azure.Storage.Queues, Version=12.14.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Azure.Storage.Queues.QueueClient+<ExistsAsync>d__51.MoveNext (Azure.Storage.Queues, Version=12.14.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener+<ExecuteAsync>d__34.MoveNext (Microsoft.Azure.WebJobs.Extensions.Storage.Blobs, Version=5.2.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Microsoft.Azure.WebJobs.Extensions.Storage.Common.Timers.TaskSeriesTimer+<RunAsync>d__14.MoveNext (Microsoft.Azure.WebJobs.Extensions.Storage.Blobs, Version=5.2.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8)
So right now there are a few different environments in play, Local developmnet, env X, dev, test and sandbox. When developing on my local machine, everything works and runs fine, no problems. X is an environment I deployed using publish in Visual Studio, works perfectly fine. Dev, test, sandbox are deployed through bicep pipelines through devops, all three of these have the same problem.
Dev, test, sandbox and X are all in the same azure subscription.
To me it looks like the function can get to the blob storage, something somewhere goes wrong with the QueueRestClient, he gets an error he can't parse and thus returns an errormessage that's not really relevant to the problem.
Code for the trigger function:
public async Task Run([BlobTrigger("drop/{name}.csv")] Stream stream, string name)
The things I've tried
It used to only be dev and test, setup at the start of the project, once I found out those weren't working I deployed sandbox when the code was fully realised, hoping (because there was no tinkering while during development) that it would magically work. But now, instead of 2 non-working environments, I have 3.
Because env X worked, I tried changing the connectionstring from the X blob storage to the dev blob storage
- X kept working fine when dropping a file.
When changing the connectionstring from dev to use the X blob storage
- Still didn't work, gave the same error.
When adding a HttpTrigger function to the same function project
- Function is pingable and works fine.
Adding a second blob trigger function which does nothing more than logging something when a file is dropped (so no complex logic)
- Same error as the "real" function.
Use 2 separate blob storage containers since one of my colleagues suggested it
- No difference, same error as before
Use managed identity instead of SAS
- No difference, same error as before
Double, tripple, quadruple check, use notepad++ text compare to make sure the connectionstrings area correct
Put connectionstrings in the configuration instead of keyvault references
- No difference, same error as before
- And purposefully putting in a wrong connectionstring tells me that the function does read the connectionstring correctly
Checked if dev and test have the same host id and lock eachother out
- Checked and not the case
Checked each and every setting between the dev function and X function
- All settings are the same except:
- Service plan settings, scale up, scale out
- Don't know enough about this to be 100% sure, but I doubt this would be the culprit
- Dev has a system assigned identity, X does not
- We're using SAS, so identity shouldn't matter and even if it'd matter, X should be the one to fail
- Service plan settings, scale up, scale out
- All settings are the same except:
Is the file that's being uploaded invalid
- The function crashes before it's even able to get to the file
Solution has been found, it didn't have anything to do with the code or the setup. Turns out the problem was that the FunctionHostID contained uppercase letters, which isn't allowed accorging to the documentation.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#azurefunctionswebhost__hostid