High speed data acquisiton using REST Services

Question

We need to develop a high speed REST based WCF Service , which will be used for updating 2000 datapoint , each data point changing at 25 msec . Is it possible to implement such high speed data acquisition using WCF

Answer 1

Using WCF yes. I'm not sure REST is the best architectural style for the type of problem you are trying to solve. I also wonder whether HTTP is appropriate.

Having said that you might want to look into CORE which is an effort to apply REST in highly constrained environments like data acquisition.

Answer 2

Here is how I am understanding your question: you expect new data values every 25 ms, or 40 x per second. There are 2000 discrete data values is one device, which means the telemetry flow from each device is around 80,000 values per second. You also have multiple devices, so your throughput will go higher than this, e.g. 800,000 updates per second for 10 devices.

In this scenario, I wouldn't expect the service layer to be a constraint, for the simple reason that it is always possible to scale up the service layer by adding more hosts to receive messages and load balancing between them. Where I would be concerned is any place where all transactions must be processed within the same domain. For example, is all this data winding up in one relational database? In that case you may have a problem with transaction throughput.

Another area that seems problematic in your architecture is the device itself. Is one device going to be capable of gathering and sending out values at 80 kHz? Here is where the REST protocol may have have too high an overhead. So it is device, not server, constraint that might drive you to find a more efficient protocol. This may be a case where writing a custom protocol directly against the socket might be warranted, but that depends on your device.