Requests from a Java client to an NLB over multiple AZs Not load-balanced
Summary:
Expectation: every request from Java is load balanced between two nodes.
Actual behavior: all the requests go to one node only, while both nodes are available.
Why is this, and what would be a best strategy to make load balancing work in this situation?
Detail:
I have a network as diagram shown below. A Java app makes an http request to an NLB. This NLB is configured over two AZs, therefore it has one endpoint for each AZ. Each AZ has an EC2 instance which listens https.
Initial Analysis:
So far, I'm aware of three issues:
- By default, an NLB endpoint forwards requests to nodes in the same AZ only. This behavior can be modified by "cross-zone load balancing" option. My concern is enabling this option might cause undesirable subeffects.
- The requests are of https, with
Keep-Alive
on. Until someone disconnects, a connection will be reused, probably causing stickness to one node. - The NLB load-balancing is made by DNS roundrobin, put aside Cross-zone load balancing mentioned in 1. Meanwhie, a Java app by default caches DNS query results. Disabling it by a JVM setting
networkaddress.cache.ttl=0
could force DNS query every time. My concern here however is that this might cause DNS flooding or something similar, which is bad for the network.
I have tried enabling "cross-zone load balancing" and adding "Connection:close" request header, to avoid 1) and 2). It seems good in an initial attempt, but not sure that is correct, especially in a larger size of request or in a high load situation etc.