I'm working in NIFI with PutDataBaseRecord to insert the data of a CSV file to a database table. Everything goes well in the first execution because there is no data in the table. Then I modify the file so it contains new records and existing ones. PutDataBaseRecord fails because of existing records (primary key constraint) but it doesn´t insert the new records.

Is there any way to configure the processor to instruct it to insert the new records and ignore the ones that failed?

I attached pictures of how my processor is configured.

Thanks in advance!

Flujo NIFI


1 Answers

Arun211 On

This is possible. However it is not a straightforward implementation.

I would suggest you to try the following flow - ListFile -> FetchFile -> SplitRecord -> PutDatabaseRecord.

In SplitRecord processor, set 'Records per Split' property to '1'.

SplitRecord processor splits up the input flow file into multiple small flow files (1 file for each row in our case due to the setting 'Records per Split=1'). Then this individual flow files will be routed to 'split' relationship i.e. to the PutDatabaseRecord processor in our flow.

PutDatabaseRecord inserts new records into table and fails for existing records.

Tested the flow with GetFile processor and it works. Hope this solves your problem. enter image description here