ADF: Copy activity suddenly stopped working with additional colums

198 views Asked by At

this is really weird: I have a copy activity that worked fine for a long time, but now stopped working and I can't understand why.

I tried many things trying to understand the issue, and I found that the additional column (a classic $$FILEPATH) is the problem.

removing the additional column everything works fine (but I can't, I need that column...)

basically copying several csv files from an SFTP to an Azure DataLake Storage V2, adding a column with the file path, now results in a very strange error:

{
    "errorCode": "2200",
    "message": "Failure happened on 'Sink' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path ingestion/cegid/dev\\.,Source=mscorlib,''Type=System.ArgumentException,Message=An item with the same key has already been added.,Source=mscorlib,'",
    "failureType": "UserError",
    "target": "CopyFromSftp",
    "details": []
}

I really don't know why it doesn't work anymore, and why it seems it's adding something already added (????). The only thig I can figure out is something has changed in the default integration runtime (maybe a bug?)

0

There are 0 answers