I am creating a logging service. The architecture is as follows: A Node.js API, that receives requests from a website (the requests can be get or post), the request is send to an SQS messaging queue, a Java worker is listening to the SQS for messages if it is post I write the data in a Cassandra database. If it is get, I read the necessary data from Cassandra, do some computations and return it to the Node.js API, which in tern returns it to the client.
The read part is a little blurry for me. Is it possible to return the data as a message in SQS? (I red that a single message can contain only 256KB of data and the read data can be more than that) I will be running multiple instances of the Node API, so is there a way to know to which instance I need to return the data? Should I create a Java API that receives read requests from the Node API (bypassing the SQS)? What is the best way to do this?
Should I use a message queue to retrieve analytics data, or should I just connect to the service that will prepare the data and receive the data from there?
You don't really return data to node.js from SQS, nodejs has to be polling the queue and needs to act on messages if/when they arrive.
SO you could use two queues, one for incoming messages (your posts), and then a second queue for your outgoing messages (your gets). But in all cases, the process that is going to consue those messages needs to be polling for them - you can't 'push' messages to a listener, the listerner needs to 'pull' them.