decrypting using openssl api with iter, default pad and salt

Question

I am trying to decrypting using openssl api functions instead of direct system command, I have encrypted through the system command using this

Edit: Solution at below answer by me.

openssl enc -aes-256-cbc -pbkdf2 -iter 310000 -md sha256 -salt -in file.run -out encrypted_data.enc -pass pass:Password!

Before this I have created my symetrickeys and used those for the above.

This is my code used for the decrypting with the system command

bool CryptoProcessor::_decryptData(Any &response)
{
  String tempPrefix = String("/tmp/aaaa-") + uuid();

  std::ifstream input(_inputFile, std::ios::binary); //the input file is the encrypted file which had signature and symetric key on top of it
  input.seekg(SIGNATURE_SIZE + KEY_SIZE, std::ios::beg);

  String encryptedDataFileName = tempPrefix + ".encrypted-data";
  OutputFileStream encryptedDataFile;
  encryptedDataFile.open(encryptedDataFileName, std::ios::binary);

  char chunk[CHUNK_SIZE];
  while (!input.eof())
  {
    input.read(chunk, CHUNK_SIZE);
    int bytesRead = input.gcount();
    encryptedDataFile.write(chunk, bytesRead);
  }
  encryptedDataFile.close();
  input.close();

  String runFileName = tempPrefix + ".run";
  String command = String("openssl enc -aes-256-cbc -d -pbkdf2 -iter 310000 -p -md sha256 -salt -pass pass:") + _symmetricKey + " -in '" + encryptedDataFileName + "' -out '" + runFileName +"'";
//I added -p to understand how the salt , key and iv are there and identify them as coming right or wrong when i use the openssl api
  logger(LOG_INFO) << command << endl;
  ANY_ASSERT(system(command.c_str()) == 0, "Decryption failed")
  return true;
}

I am trying like this to decrypt it using api but code succcesfully decrypts the file but it is giving the wrong file i.e decrypting not done properly, I checked the salt, key and IV of these in the below code, i found those are very different and not the expected.

    // Create an OpenSSL BIO object for input and output
    BIO *bio_in = BIO_new_file(encryptedDataFileName.c_str(), "rb");
    BIO *bio_out = BIO_new_file(runFileName.c_str(), "wb");

    if (!bio_in || !bio_out)
    {
        logger(LOG_ERROR) << "Failed to open input/output BIOs" << std::endl;
        return false;
    }

    // Create an EVP decryption context
    EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
    if (!ctx)
    {
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to create cipher context" << std::endl;
        return false;
    }

    // Set the AES-256-CBC cipher
    if (EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, nullptr, nullptr) != 1)
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to initialize decryption context" << std::endl;
        return false;
    }

    EVP_CIPHER_CTX_set_padding(ctx, 0); // Disable padding

    // Set the symmetric key using PBKDF2
    unsigned char derivedKey[32]; // 256 bits for AES-256
    int iterCount = 310000;
    logger(LOG_INFO) << "Iterations: " << iterCount << std::endl;
    const char *passphrase = _symmetricKey.c_str();
    logger(LOG_INFO) << "Passphrase: " << _symmetricKey << std::endl;
    unsigned char salt[PKCS5_SALT_LEN];
    RAND_bytes(salt, sizeof(salt))

    unsigned char iv[16];
    RAND_bytes(iv, sizeof(iv));

    if (PKCS5_PBKDF2_HMAC(passphrase, -1, salt, sizeof(salt), iterCount, EVP_sha256(), 32, derivedKey) != 1)
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to derive encryption key using PBKDF2" << std::endl;
        return false;
    }
    logger(LOG_INFO) << "Derived Key: ";
    for (int i = 0; i < 32; ++i)
    {
        logger(LOG_INFO) << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(derivedKey[i]);
    }
    logger(LOG_INFO) << std::dec << std::endl;

    // Set the key and IV for decryption
    if (EVP_DecryptInit_ex(ctx, nullptr, nullptr, derivedKey, nullptr) != 1)
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to set decryption key and IV" << std::endl;
        return false;
    }

Help me, In writing the code for the given system command of openssl with api.

Answer 1

Decryption MUST use the same values for salt and IV as encryption used, but you're generating new random values which will be wrong with overwhelming probability. openssl enc stores the salt at offset 8-15 in the file (read it from there), and only offset 16-last is actually ciphertext (decrypt only that part); it also derives both key and IV (if the mode uses it, which CBC does) from password+salt, so do the PBKDF2 call to generate 48 bytes and use the first 32 for the key and the last 16 for the IV. Also for a block mode like CBC enc by default uses PKCS5/7 padding, so by turning off padding during decryption you will add some garbage bytes to your plaintext.

PS: if you declare inBuf outBuf as array of unsigned char (not just char) you don't need to the ugly casts, and as a bonus it more accurately describes what is in at least the encrypted file and potentially the decrypted one, namely raw data that is not actually characters.

Answer 2

Thanks @dave_thompson_085 The message from him helped me solving this, This is the solution when you are encryption is having salt, Iter and default pad value.

bool CryptoProcessor::_decryptData(Any &response)
{
    String tempPrefix = String("/tmp/aaaa-") + uuid();

    std::ifstream input(_inputFile, std::ios::binary);
    input.seekg(SIGNATURE_SIZE + KEY_SIZE, std::ios::beg);

    String encryptedDataFileName = tempPrefix + ".encrypted-data";
    OutputFileStream encryptedDataFile;
    encryptedDataFile.open(encryptedDataFileName, std::ios::binary);

    unsigned char chunk[CHUNK_SIZE];
    
    String runFileName = tempPrefix + ".run";


    // Create an EVP decryption context
    EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
    if (!ctx)
    {
        logger(LOG_ERROR) << "Failed to create cipher context" << std::endl;
        return false;
    }

    // Set the AES-256-CBC cipher
    if (EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, nullptr, nullptr) != 1)
    {
        EVP_CIPHER_CTX_free(ctx);
        logger(LOG_ERROR) << "Failed to initialize decryption context" << std::endl;
        return false;
    }

    // Create BIOs using file pointers
    FILE *inFile = fopen(encryptedDataFileName.c_str(), "rb");
    FILE *outFile = fopen(runFileName.c_str(), "wb");

    BIO *bio_in = BIO_new_fp(inFile, BIO_NOCLOSE);
    BIO *bio_out = BIO_new_fp(outFile, BIO_NOCLOSE);

    if (!bio_in || !bio_out)
    {
        logger(LOG_ERROR) << "Failed to create input/output BIOs" << std::endl;
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        return false;
    }


    // Derive the key using PBKDF2
    unsigned char derivedKey[32]; // 256 bits for AES-256
    //unsigned char encryptionKey[KEY_SIZE]; 
    int iterCount = 310000;
    const char *passphrase = _symmetricKey.c_str();
    logger(LOG_INFO) << passphrase << std::endl;

    unsigned char salt[PKCS5_SALT_LEN];
    unsigned char block[16]; // Read the full 16 bytes
    if (BIO_read(bio_in, block, sizeof(block)) != sizeof(block))
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to read salt from input file" << std::endl;
        return false;
    }

    // Extract the last 8 bytes as the salt
    memcpy(salt, block + 8, PKCS5_SALT_LEN);

    const EVP_CIPHER *cipher = EVP_aes_256_cbc();
    int iklen = EVP_CIPHER_key_length(cipher);
    int ivlen = EVP_CIPHER_iv_length(cipher);

    unsigned char key[iklen];
    unsigned char iv[ivlen];
    unsigned char keyivpair[iklen + ivlen];

    // Derive the key and IV using PBKDF2
    if (!PKCS5_PBKDF2_HMAC(passphrase, -1, salt, sizeof(salt), iterCount, EVP_sha256(), iklen + ivlen, keyivpair))
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to derive encryption key using PBKDF2" << std::endl;
        return false;
    }

    // Copy the derived key and IV to their respective buffers
    memcpy(key, keyivpair, iklen);
    memcpy(iv, keyivpair + iklen, ivlen);

    // Set the key and IV for decryption
    if (EVP_DecryptInit_ex(ctx, cipher, NULL, key, iv) != 1)
    {
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        logger(LOG_ERROR) << "Failed to set decryption key and IV" << std::endl;
        return false;
    }

    //Use for checking the salt and IV values if anything goes wrong, verify using the command line by passing arg -p to compare the values of the below
    // printf("Salt: ");
    // for (int i = 0; i < sizeof(salt); ++i)
    // {
    //     printf("%02X", salt[i]);
    // }
    // printf("\n");

    // printf("Derived Key: ");
    // for (int i = 0; i < sizeof(derivedKey); ++i)
    // {
    //     printf("%02X", derivedKey[i]);
    // }
    // printf("\n");

    // printf("IV: ");
    // for (int i = 0; i < sizeof(iv); ++i)
    // {
    //     printf("%02X", iv[i]);
    // }
    // printf("\n");

    // Perform the decryption
    unsigned char inBuf[CHUNK_SIZE];
    unsigned char outBuf[CHUNK_SIZE];

    int outLen;
    while (true)
    {
        int bytesRead = BIO_read(bio_in, inBuf, CHUNK_SIZE);
        if (bytesRead <= 0)
            break;

        if (EVP_DecryptUpdate(ctx, outBuf, &outLen, inBuf, bytesRead) != 1)
        {
          
            EVP_CIPHER_CTX_free(ctx);
            BIO_free_all(bio_in);
            BIO_free_all(bio_out);
            logger(LOG_ERROR) << "Decryption failed" << std::endl;
            
            return false;
        }

        BIO_write(bio_out, outBuf, outLen);
    }

    if (EVP_DecryptFinal_ex(ctx, outBuf, &outLen) != 1)
    {
        unsigned long error = ERR_get_error();
        char errorString[256];
        ERR_error_string_n(error, errorString, sizeof(errorString));
        logger(LOG_ERROR) << "Decryption finalization failed: " << errorString << std::endl;
        EVP_CIPHER_CTX_free(ctx);
        BIO_free_all(bio_in);
        BIO_free_all(bio_out);
        return false;
    }

    BIO_write(bio_out, outBuf, outLen);

    // Clean up
    EVP_CIPHER_CTX_free(ctx);
    BIO_free_all(bio_in);
    BIO_free_all(bio_out);

    logger(LOG_INFO) << "Decryption Done" << std::endl;
    remove(encryptedDataFileName.c_str());
    
    return true;
}