How to handle EAGAIN case for TLS over SCTP streams using memory BIO interface

Question

I'm using BIO memory interface to have TLS implemented over SCTP.

So at the client side, while sending out application data,

1. SSL_write() api encrypts the data and writes data to the associated write BIO interface.
2. Then the data from BIO interface is read to a output buffer using BIO_read() call and then
3. send out to the socket using sctp_sendmsg() api.

Similarly at the server side, while reading data from socket

1. sctp_recvmsg() api reads ecrypted message chunks from socket,
2. BIO_write() api writes it to the read BIO buffer, and
3. SSL_read() api decrypts the data read from the BIO.

The case i'm interested at is where at client side, steps 1 and 2 are done, and while doing 3, i get an EAGAIN from the socket. So whatever data i've read from the BIO buffer, i clean it up, and ask application to resend the data again after some time.

Now when i do this, and later when steps 1, 2 and 3 at client side goes through fine, at the server side, openssl finds it that the record that it received has got a a bad_record_mac and closes the connection.

From googling i came to know that one possibility for it to happen is if TLS packets comes out of sequence, as MAC encoding has dependency on the previous packet encoded, and, TLS needs to have the packets delivered in the same order. So when i was cleaning up the data on EAGAIN i am dropping an SSL packet and then sending a next packet which is out of order (missing clarity here) ?

Just to make sure of my hypothesis, whenever the socket returned EAGAIN, i made the code change to do an infinite wait till the socket was writeable and then everything goes fine and i dont see any bad_record_mac at server side.

Can someone help me here with this EAGAIN handling ? I can't do an infinite wait to get around the issue, is there any other way out ?

Answer 1

1. Select for writability.
2. Repeat the send.
3. If the send was incomplete, remove the part of the buffer that got sent and go to (1).

So whatever data i've read from the BIO buffer, i clean it up

I don't know what this means. You're sending, not receiving.

Just to make sure of my hypothesis, whenever the socket returned EAGAIN, i made the code change to do an infinite wait till the socket was writeable and then everything goes fine and i dont see any bad_record_mac at server side.

That's exactly what you should do. I can't imagine what else you could possibly have been doing instead, and your description of it doesn't make any sense.

Answer 2

... i get an EAGAIN from the socket. So whatever data i've read from the BIO buffer, i clean it up, and ask application to resend the data again after some time.

If you get an EAGAIN on the socket you should try to send the same encrypted data later.

What you do instead is to throw the encrypted data away and ask the application to send the same plain data again. This means that these data get encrypted again. But encrypting plain data in SSL also includes a sequence number of the SSL frame and this sequence number is not the same as for the last SSL frame you throw away.

Thus, if you have thrown away the full SSL frame you are trying to send a new SSL frame with the next sequence number which does not fit the expected sequence number. If you've succeeded to send part of the previous SSL frame and thew away the rest then the new data you send will be considered part of the previous frame which means that the HMAC of the frame will not match.

Thus, don't throw away the encrypted data but try to resent these instead of letting the upper layer resent the plain data.