I have configured a Stream Analytics Job to write output to a Storage Blob Container with an immutability policy. According to the provided documentation (link), with an immutable storage container, a new file should be created for every write operation. However, when I attempted this setup, instead of creating new files, the job encountered failures, and only the initial file was generated.
Here are the specifics of my query and job configuration:
SELECT *
INTO [blob-output]
FROM [eventhub-input];
The output is a line-separated JSON blob written to a storage container with a time-based retention immutability policy set for 10 days. The path pattern is defined as type={type}/year={datetime:yyyy}/month={datetime:MM}/day={datetime:dd}/hour={datetime:HH}/. I'm using a single streaming unit, and the job's minimum rows are set to 1,000 with a maximum time of 5 minutes. Write mode is set to Append because Once cannot be used with a custom partition key. Hence, I'm attempting to utilize the container immutability feature.
However, upon execution, I encounter the following error:
OutputProcessorFailure, We are experiencing issues writing output for output blob-output right now. We will try again soon.
Send Events: Output processor failure
Interestingly, upon removing the immutability policy, the job functions as expected, and I can observe modifications to existing files.
Could anyone provide insight into why the Stream Analytics Job fails to create new files in an immutable storage blob container as expected, and how to rectify this issue?