Bad MAC after porting OpenSSL 1.0.2 to ECOS

Question

We have a OpenSSL running on our embedded system, which is running ECOS OS. We are now upgrading our OpenSSL to 1.0.2 version. We have successfully ported and compiled the OpenSSL library. But when when we try to connect our device using SSL (via https), handshake fails with bad record mac alert always. We have enabled OpenSSL debug option, but unable to identify why its failing.

Have someone ported latest OpenSSL code to ECOS? Do we need to take of any special compilation flags with latest OpenSSL code for ECOS?

For reference, here is the relevant part of ssl3_get_record:

mac = rr->data + rr->length;
i=s->method->ssl3_enc->mac(s,md,0 /* not send */);
if (i < 0 || CRYPTO_memcmp(md, mac, (size_t)mac_size) != 0)
    {
    al=SSL_AD_BAD_RECORD_MAC;
    SSLerr(SSL_F_SSL3_GET_RECORD,SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC);
    goto f_err;
    }

Answer 1

After debugging we found that the random library (RAND) was failing for ECOS. There were lot of places in OpenSSL where it checks for random_bytes return type. Due to this failure, pre-master key decryption was failing. And incoming packets were not decrypted properly. Hence a BAD Mac records error was seen.

We also checked with our old ported code (0.9.6), RAND library was failing there also, but there we no return check for random_bytes and pseudo_rand_bytes. As a fix we made RAND to return success every time, and we can see SSL session being established fine with OpenSSL 1.0.2 version.