Our problem: we have more than 300 000 online clients to application (50 000 to one node). Because we use load balancer we don't now where (which nginx instance) alive connection to client. So we need pub message to all nodes. How we can get linear scalability?
Current functionality:
- Comet(for us comet is required) clients (mobile/web browsers) and server (java) with pub-sub functionality.
- Client can send notifications to other (single) client;
- Server can send notifications to single client or group of clents;
- Clients recieve information about active clients (by group) from nginx+memcached
Now we use: nginx + https://github.com/wandenberg/nginx-push-stream-module + memcached.
Client before set up on comet channel send request to server: "I am alive and my commet channel id ...". Each 10 min client reopen pipeline, but register as alive on server only once (it's other problem).
I think we can write some logic with Lua on nginx(+redis or memcached). And each client request for reopen pipeline update information in redis (+add additional information about nginx instance ip which bind connection). But may be exist other variants or practice for scalability comet functionality?