How does Digg's streaming API work?

Question

I really like this: http://services.digg.com/2.0/stream

Data is continuously received using a single connection, content-type is application/json. Also, it's consumable by a browser. How is this implemented? Looking around a bit it does not seem to be long-polling, HTML5 server-sent events or even websockets.

How would I go about building something like that? Are there any open-source solutions I can use as a starting point?

Thanks,

/David

Answer 1

It's just a stream of bytes transmitted with chunked transfer encoding. Basically it's just HTTP chunked streaming.

Since you mentioned server-sent events, you can think of this as SSE without the special formatting. Messages are JSON objects delimited by newline. On the server you can implement this the same way you would implement SSE minus formatting.

A client for such an API needs to be able to parse such a stream of data and split messages on newline. Some browsers support a readyState of 4 on the XMLHttpRequest, which is called for each chunk, thus allowing you to process the streamed data.