Decryption with invalid key not always throws CryptographicException

Question

I'm using following settings for my crypto-service

        var cipher = new RijndaelManaged();
        cipher.KeySize = 256;
        cipher.BlockSize = 256;
        cipher.Padding = PaddingMode.ISO10126;
        cipher.Mode = CipherMode.CBC;
        cipher.Key = _key;
        return cipher;

For my service I wrote test which tries to decrypt encrypted data with invalid key.

    [Test]
    public void EncryptDecrypt_UsingInvalidKey_ShouldThrowCryptographicException()
    {
            var testInput = new TestInput();
            var encriptedData = _service.Encrypt(testInput.Input);
            var decrtryptServiceWithNewKey = GetService();

            Assert.Throws<CryptographicException>(() => decrtryptServiceWithNewKey.Decrypt(encriptedData));

    }

Th problem is that sometimes this test fails due to it doesn't throw CryptographicException(Padding is invalid and cannot be removed.) which I expected.

Initial Encryption Key :d/Ww5u5V/dV4FzYaFp8QEKeikFEFxaK9Z9ZBER1/y5A=

case 0) try decrypt with key :2pPmUsE4Erk+kUQPnmDCBzVlvvvePIjGJpi3cacJy1U=
                          iv :Hi4GbhuSKR5UUTyCgO+VNb/jQ+U1SfxXPPiAgiyCRhM=

  1)   key :40IeO1ZJF6miVFfmEaTkJDCus/EeiWBzzFqhvlxg/Js=
        iv :B535OzpMErNeSVeR98q5YEIikh0juk9YIsOBWop/N0I=
                                              Test pass

  2)   key :d1M0Mq4/5JIcp5oF8nvfj1e8hHv5uo0zSMf820ThnkM=
        iv :LAGLmg0LORGpHRWMKw3myPzuUjqlH4urn4+iMl248CE=
                                              Test pass

  3)   key :jDJdtNBUHe0CeNwf4VdyqpdMOAiDBFYZ0XJIXjw9BOw=
        iv :bCzj4E2jFkb3CRY5J6K1YjzsEBSn2ULjHNXtbZIbk2o=
                                              Test pass

  4)   key :zCbD4krNVS1wg8kjcW4+B7esMeJhCFK3PrlMrphpC0s=
        iv :rjLqP2L52KKu09tzJEuuIV9vRxpURI/Fz0R9vmOjsw8=

 Expected: <System.Security.Cryptography.CryptographicException>
 But was:  null

Could you please suggest why in some cases I can't get CryptographicException in my test?

Answer 1

The problem is that the decrypted plaintext has the correct padding about once in eight times. This happens when the decrypted plaintext ends with 01 to 20. The block size of 256 / 32 bytes makes for a valid padding for that particular range.

Padding should never be used to verify correctness of ciphertext/plaintext. It is even dangerous to do so as it may lead to padding oracle attacks. Instead you should use a (H)MAC over the ciphertext - adding an authentication tag to the ciphertext. A MAC should use a different key and should also include the IV. It is also possible to use a modern authenticated cipher such as GCM which allows using the same key - GCM/CCM/EAC etc already include the IV in the authentication tag.

CBC mode encryption on it's own can only be used for confidentiality, and that only when padding/plaintext oracle attacks are not applicable (i.e. it should never be used as is in a transport protocol). An authentication tag is required to add integrity (what you are currently trying to test) and authenticity.

Note that ISO10126 is obsolete and that Rijndael with a block size of 256 bits is not AES. So you are not using any standardized crypto at the moment.