Azure Data factory expression

Question

i saw this expression in one of the Pipeine , in the filter activity condition. can anyone help me understad this expresson used( you can refrase inorder to make it understandable). looks difficult to understand.

@if(equals(pipeline().parameters.FileName,'default'),endswith(toUpper(item().name),'.PDF'),
and(startswith(item().name,replace(string(pipeline().parameters.Filemane),'*.txt','')),
endswith(toUpper(item().name),'.PGP')))

Thanks

I dont have blocker , but i cannot understand the Expression. Just wanted to get some clarity what is the purpose of that code , what are they trying to achieve in that particualar filter condition in the ADF

Answer 1

If you use the above expression in the filter activity. It will filter the values of the array based on the FileName parameter.

This is my sample array of file names:

['rakesh.pdf','correct1.pgp','wrong1.pgp','laddu.pdf','correct2.pgp','wrong2.pgp','virat.pdf','wrong3.pgp','correct3.txt']

For you the array will be an array of objects. that's why in the above expression it's there as item().name. For me it's only item().

Filter activity filters based on the condition. If the particular array item satisfies the condition(true/false) then it filters it.

 @if(equals(pipeline().parameters.FileName,'default'),endswith(toUpper(item().name),'.PDF'), and(startswith(item().name,replace(string(pipeline().parameters.Filemane),'*.txt','')), endswith(toUpper(item().name),'.PGP')))

In the above expression, If the FileName parameter value is 'default', it filters the items which are ending with .PDF. if(condition,True case,else case) so in true case we are checking endswith() .PDF or not. If it is true, then filter condition is true and that particular array value will be filtered.

else case

and(startswith(item().name,replace(string(pipeline().parameters.Filemane),'*.txt','')), endswith(toUpper(item().name),'.PGP'))

If the parameter value is correct*.txt(other than default). In this case it replaces the '*.txt' with empty strings('') and returns 'correct'. Then it checks(first condition) the whether the array value starts with 'correct' and endswith .PGP(second condition). If both conditions are true, then filter condition is true and array value is filtered.

This is a sample demo:

Array:

Filter with same condition:

for me it's only item() not item().name.

If I take the parameter value as 'default'.

Filter output array (all .PDF files):

If I take the parameter value as 'correct*.txt'.

Filter output array(.PGP files starts with 'correct'):

NOTE: If your parameter value is only like 'correct.txt' then use '.txt' in the expression.

Answer 2

Well I am not sure if there is a difference between the first code before 'Thanks' and the code after it.

But it seems that the code will return true or false. I can reformulate to this in c# like format:

if(pipeline().parameters.FileName == 'default')
{
    // if the name of item ends with .pdf then return true, else return false
    return endswith(toUpper(item().name),'.PDF'), 
}
else
{
   // replace the *.txt with an empty string. 
   // I think it means if the file ends with .txt then replace it with an empty string
   string replacedText = replace(pipeline().parameters.Filemane,'*.txt','')
   // check if the itemName is ends with .txt (in this case this condition will fail)
   boolean cond1 = startswith(item().name, replacedText)
   
   // if the name of item ends with .PGP then cond2 = true
   boolean cond2 = endswith(toUpper(item().name),'.PGP')
   
   return cond1 && cond2 
}