How to dynamically place Azure Data flow output in folders named after date

Question

I have a dataflow in a pipeline that outputs a json file. Every time it runs I want the json file to be placed in a folder that is name the dynamically the current time stamp down to the second that the folder is pipeline is ran. I have tried put a utcNow function in the file path setup of the output dataset dataset folderpath setup screenshot. But I get a

"Cannot create through WASB that has colons in the name" error.

Even though I have created folders with colons in the name through Azure storage before. Please help.

Answer 1

I tested that set folder name with colons and expression in Data Flow.

No matter we set the folder name with colons or expression, We will get the error:

{"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: JSON parsing error, unsupported encoding or multiline","Details":"at Sink 'sink1': org.apache.spark.SparkException: Job aborted."}

According the error message, folder name with colons and expression function are not supported in Data Flow.

It only works in Copy active sink dataset.