Invalid characters generated while encrypting with openssl evp aes_256_ctr() mode

Question

My idea is to do file encryption in a client server model and i am using openssl evp for encryption purpose. I need to store the cipher text in a text file and send it to the client. But i am unable to do this because i find invalid characters being present in the cipher text which cannot be stored in a file.

This is my code for encryption :

EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit_ex(&ctx, EVP_aes_256_ctr(), NULL, NULL, NULL,
        do_encrypt);
OPENSSL_assert(EVP_CIPHER_CTX_key_length(&ctx) == 32);
OPENSSL_assert(EVP_CIPHER_CTX_iv_length(&ctx) == 16);

EVP_CipherInit_ex(&ctx, NULL, NULL, key, iv, do_encrypt);

//receive the file contents in chunks of 1024 bytes
while ((inlen = recv(connfd, inbuf, sizeof inbuf, 0)) > 0) {
    fprintf(stdout,"\nReceived %d bytes",inlen);
    fflush(stdout);
    fprintf(stdout,"\nOriginal: %s",inbuf);
    fflush(stdout);
    //use encrypt_update() to encrypt the chunks
    if(!EVP_CipherUpdate(&ctx, outbuf, &outlen, inbuf, inlen)) {
        /* Error */
        EVP_CIPHER_CTX_cleanup(&ctx);
        return 0;
    }
    //write the encrypted text to out file
    fprintf(stdout,"\nEncrypted: %s %d",outbuf, inlen);
    fflush(stdout);
    fwrite(outbuf, sizeof(char), outlen, fp);
    //clear the buffer
    memset(inbuf,0, strlen(inbuf));
    memset(outbuf,0, strlen(outbuf));
}
//use encrypt_final() to encrypt the final letf out block of chunk is any
if(!EVP_CipherFinal_ex(&ctx, outbuf, &outlen)) {
    /* Error */
    EVP_CIPHER_CTX_cleanup(&ctx);
    return 0;
}
//write the encrypted text to out file
fwrite(outbuf, sizeof(char), outlen, fp);
EVP_CIPHER_CTX_cleanup(&ctx); //cleanup
fclose(fp); //close the file

I referred this link where an issue of invalid characters with decryption being reported and solved.

Issues with encrypting a file using openssl evp api(aes256cbc)

I hope someone could help me out here.

Thanks in advance.

Answer 1

Invalid characters generated while encrypting with openssl evp aes_256_ctr() mode...

... because i find invalid characters being present in the cipher text which cannot be stored in a file

I think this is your problem. Its not quite correct.

You can store anything (any character) in a file. C-strings are a little different, but you are not working with a string.

All characters are equally probably in the cipher text (equally probable as any other character, like 0x00, 0x01, ... 'A', 'B', ... 'a', 'b', ..., 0xFE, 0xFF).

---

fprintf(stdout,"\nOriginal: %s",inbuf);

This could be a problem if inbuf has an embedded NULL. I thought you were dealing with files and not strings?

---

memset(inbuf,0, strlen(inbuf));
memset(outbuf,0, strlen(outbuf));

As Iridium said, these are not needed. You should be using the return values of functions like recv (and not depending on distinguished characters like NULL since its equally probably in the cipher text (equally probable as any other character, like 0x00, 0x01, ... 'A', 'B', ... 'a', 'b', ..., 0xFE, 0xFF).

---

EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit_ex(&ctx, EVP_aes_256_ctr(), NULL, NULL, NULL, do_encrypt);
...

Your also ignoring return values. That's usually a bad idea.