C# .net core encryption

Question

I'm trying to use a public key of a user to encrypt a string in a sort of pgp fashion, but I keep getting the error:

bignum routines:BN_mod_inverse:no inverse

I've looked around and I cannot find anything specific as to what I'm doing wrong. I've looked around for .NET core information, but I cannot seem to find anything relevant.

I'm using the following code:

byte[] publicKey = Encoding.UTF8.GetBytes(key);

RSA rsa = RSA.Create();
RSAParameters RSAKeyInfo = new RSAParameters();

RSAKeyInfo.Modulus = publicKey;
RSAKeyInfo.Exponent = new byte[]{1,0,1};

rsa.ImportParameters(RSAKeyInfo);
var encrypted = rsa.Encrypt(Encoding.UTF8.GetBytes(user.challenge.text), RSAEncryptionPadding.Pkcs1);

It's entirely possible I'm going about this entirely wrong, so any thoughts or suggestions would be great!

Answer 1

Your overall structure (build RSAParameters, call ImportParameters, call Encrypt) is valid, which suggests that your error is in Modulus recovery.

If your Modulus is input as a string, it's likely encoded as (most to least)

• Base64 (Convert.FromBase64String)
• Hex (May need a manual parser)
• UTF-8

UTF-8 is really unlikely, since the Modulus value can contain bytes whose value are 0 (and other invalid/unexpected UTF-8 sequences). While all even-length byte sequences encoded as hex can be validly decoded as Base64, it's extraordinarily unlikely that you'd misinterpret them given two or three different inputs.

Other noteworthy things:

• RSA is IDisposable, you should put it in a using statement to ensure resources free up earlier when you are done with them.
• The Exponent is usually 0x010001, but that isn't required. Unless you have a guaranteed constraint on it, you should be serializing it, too.
  • And if it is guaranteed to always be 0x010001, why make a new one each time? Save it as a static field and make the GC's job easier.
• As Maarten said, RSA-ENC-PKCS1 is susceptible to a padding oracle attack, so (especially if your data is over the wire) you should use OAEP.
  • In the context of .NET, OaepSHA1 has the best support (all inbox providers). OAEP with a SHA-2 algorithm is only supported by RSACng (or the opaque RSA.Create() on Windows).