I'm trying to program custom RSA key pair generation algorithm using OpenSSL. I've used the PKCS5_PBKDF2_HMAC_SHA1
function to generate PRNG seed, so, I've used this seed as RAND_seed input.
Unfortunately every time I call RAND_bytes
, with the same seed, I obtain different random numbers, but this isn't the expected behaviour, because as say the answer at How can one securely generate an asymmetric key pair from a short passphrase? the random number generator is deterministic (same seed same output).
Below is the test case. I've declared also constant seed, but the generation is never deterministic.
unsigned int seed = 0x00beef00;
unsigned int rnum[5];
RAND_seed(&seed, sizeof(seed));
RAND_bytes((unsigned char *)&rnum[0], sizeof(rnum));
Where is the error?
Openssl's
int RAND_bytes(unsigned char *buf, int num);
tries to make things as random as it can. That is apparently a feature you don't want, and are instead looking for a repeatable pseudorandom sequenceBut Openssl also has
int RAND_pseudo_bytes(unsigned char *buf, int num);
which is probably what you are looking for, the pseudo part, to give you the repeatable sequence.
https://linux.die.net/man/3/rand_pseudo_bytes
Force openssl's RNGs to return a repeatable byte sequence
As an aside, if you are doing RSA, a repeatable random sequence isn't all that good, because what you are looking for are two large primes. Doing a large PRNG number, then testing for prime is probably a bad idea. Half of them will be divisible by 2! :)
If I recall correctly, you need to pick a good starting number, make it odd by ORing with 1, then test for prime, and if it is not, increment by 4 and try again.
Once you find them, you probably will not want to do it again using repeatable PRNG bytes to start your search.
And while I'm pretty sure this is for a learning project, but if all you want is a RSA key pair from openssl check out
https://stackoverflow.com/a/5246045/6047952