We have a streaming application(IBM Infosphere Streams),wherein the input to the application is from the input_queue(JMS queue) and the output is again written to some output_queue(JMS Queue). Currently we are talking about messages in excess of 300k per hour, But at times,the application gets clogged(due to the limited resources availabe) and the messages are queued internally by the application and the output_queue doesn't get the messages at the same rate. So fix this ,we have no option but to inhibit the ingestion of messages from input_queue into the streams application ,and restart the application with ingestion enabled. During this time,the internally queued message gets lost.So we are looking forward to recover the lost messages in case streams is required to be stopped. One approach we thought of is,to have another queue which takes in messages within streams while it ingest them. and and removes them once they are posted to output queue and in case streams have to be stopped,we can replay those message in the internal queue first and then begin processing other messages from the input_queue.
Now The question is how efficient this approach is and if possible can any one suggest workarounds. I have been told that posting to the queue over network is an expensive operation.So what options do we have.Suggestions are most welcome .Thanks.