Send data from Node.js API through SQS to Java worker and return the result of the worker to the API through SQS

209 views Asked by At

I am creating a logging service. The architecture is as follows: A Node.js API, that receives requests from a website (the requests can be get or post), the request is send to an SQS messaging queue, a Java worker is listening to the SQS for messages if it is post I write the data in a Cassandra database. If it is get, I read the necessary data from Cassandra, do some computations and return it to the Node.js API, which in tern returns it to the client.

The read part is a little blurry for me. Is it possible to return the data as a message in SQS? (I red that a single message can contain only 256KB of data and the read data can be more than that) I will be running multiple instances of the Node API, so is there a way to know to which instance I need to return the data? Should I create a Java API that receives read requests from the Node API (bypassing the SQS)? What is the best way to do this?

Should I use a message queue to retrieve analytics data, or should I just connect to the service that will prepare the data and receive the data from there?

1

There are 1 answers

1
E.J. Brennan On

You don't really return data to node.js from SQS, nodejs has to be polling the queue and needs to act on messages if/when they arrive.

SO you could use two queues, one for incoming messages (your posts), and then a second queue for your outgoing messages (your gets). But in all cases, the process that is going to consue those messages needs to be polling for them - you can't 'push' messages to a listener, the listerner needs to 'pull' them.