count of records in a streamsets stage

Question

I use Streamsets to ingest records from oracle to ElasticSearch. I want to register in maprDB destination the count of record that I process each step in my oracle query. How can I get the number of records at a certain streamsets stage?

Answer 1

You can insert one of the script processors (JavaScript Evaluator) to trigger event record. And connect MaprDB destination to script processor event lane.

var eventRecord = sdcFunctions.createEvent("count", 1);
eventRecord.value = { recordCount :  records.length};
sdcFunctions.toEvent(eventRecord);

Sample Pipeline