I'm testing SSL/TLS stream proxying within NGINX that will connect to a web server using gnutls as the underlying TLS API. Using the command line test tool in gnutls (gnutls-serv) the entire process works, but I can't understand the logic:
the NGINX client (proxying HTTP requests from an actual client to the gnutls server) seems to want to handshake the connection multiple times. In fact in most tests it seems to handshake 3 times without error before the server will respond with a test webpage. Using wireshark, or just debugging messages, it looks like the socket on the client side (in the perspective of the gnutls server) is being closed and reopened on different ports. Finally on the successful connection, gnutls uses a resumed sessions, which I imagine is one of the previously mentioned successful handshakes.
I am failing to find any documentation about this sort of behaviour, and am wondering if this is just an 'NGINX thing.'
Though the handshake eventually works with the test programs, it seems kind of wasteful (to have multiple expensive handshakes) and implementing handshake logic in a non-test environment will be tricky without actually understanding what the client is trying to do.
I don't think there are any timeouts or problems happening on the transport, the test environment is a few different VMs on the same subnet connected between 1 switch.
NGINX version is the latest mainline: 1.11.7. I was originally using 1.10.something, and the behaviour was similar though there were more transport errors. Those errors seemed to get cleaned up nicely with upgrading.
Any info or experience from other people is greatly appreciated!
Use either RSA key exchange between NGINX and the backend server or use SSLKEYLOGFILE LD_PRELOAD for NGINX to have the necessary data for Wireshark to decrypt the data.
While a single incoming connection should generate just one outgoing connection, there may be some optimisations in NGINX to fetch common files (favicon.ico, robots.txt).