ExecutionScript output two different flowfiles NIFI

Question

I'm using executionScript with python and I'm having a dataset which it may have some corrupted data, my idea is to process the good data, and put it in my flowfile content to my success relationship and the corrupted one redirect them in the failure relationship, I have done something like this :

for msg in messages :
   try :
        id = msg['id']
        timestamp = msg['time']
        value_encoded = msg['data']
        hexFrameType = '0x'+value_encoded[0:2]          
        matches = re.match(regex,value_encoded)
        ....
    except:
        error_catched.append(msg)

        pass

any idea how can I do that ?

Answer 1

For the purposes of this answer I am assuming you have an incoming flow file called "flowFile" which you obtained from session.get(). If you simply want to inspect the contents of flowFile and then route it to success or failure based on an error occurring, then in your success path you can use:

session.transfer(flowFile, REL_SUCCESS)

And in your error path you can do:

session.transfer(flowFile, REL_FAILURE)

If instead you want new files (perhaps one containing a single "msg" in your loop above) you can use:

outputFlowFile = session.create(flowFile)

to create a new flow file using the input flow file as a parent. If you want to write to the new flow file, you can use the PyStreamCallback technique described in my blog post.

If you create a new flow file, be sure to transfer the latest version of it to REL_SUCCESS or REL_FAILURE using the session.transfer() calls described above (but with outputFlowFile rather than flowFile). Also you'll need to remove your incoming flow file (since you have created child flow files from it and transferred those instead). For this you can use:

session.remove(flowFile)