this is really weird: I have a copy activity that worked fine for a long time, but now stopped working and I can't understand why.
I tried many things trying to understand the issue, and I found that the additional column (a classic $$FILEPATH) is the problem.
removing the additional column everything works fine (but I can't, I need that column...)
basically copying several csv files from an SFTP to an Azure DataLake Storage V2, adding a column with the file path, now results in a very strange error:
{
"errorCode": "2200",
"message": "Failure happened on 'Sink' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path ingestion/cegid/dev\\.,Source=mscorlib,''Type=System.ArgumentException,Message=An item with the same key has already been added.,Source=mscorlib,'",
"failureType": "UserError",
"target": "CopyFromSftp",
"details": []
}
I really don't know why it doesn't work anymore, and why it seems it's adding something already added (????). The only thig I can figure out is something has changed in the default integration runtime (maybe a bug?)