How does OpenSSL choose which ENGINE to use?

Question

I have an application that dynamically links with OpenSSL 1.0.2 and TPM hardware with OpenSSL ENGINE implementation for RSA.

I use OpenSSL's dynamic ENGINE to register the TPM ENGINE. This is how the (simplified) code looks:

ENGINE_load_dynamic();
ENGINE *e = ENGINE_by_id("dynamic");

ENGINE_ctrl_cmd_string(e, "SO_PATH", path_to_libtpm, 0);
ENGINE_ctrl_cmd_string(e, "ID", "tpm2tss", 0);
ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);

ENGINE_init(e);

ENGINE_ctrl_cmd(e, ...);
ENGINE_ctrl_cmd(e, ...);

ENGINE_register_all_complete();

ENGINE_finish(e);
ENGINE_free(e);

According to the man page, since I'm calling ENGINE_register_all_complete() instead of ENGINE_set_default_RSA, I am letting OpenSSL decide which implementation of RSA to use.

the next time OpenSSL tries to set up an RSA key, any bundled ENGINEs that implement RSA_METHOD will be passed to ENGINE_init() and if any of those succeed, that ENGINE will be set as the default for RSA use from then on

Will OpenSSL prioritize RSA implementation in a registered ENGINE over its own implementation?

What happens when there are several ENGINEs registered that provide implementations for the same algorithm? Will OpenSSL use the first ENGINE it is able to initialize?

Is there any guarantee that a registered ENGINE will be used if ENGINE_set_default_XXX is not called?

Answer 1

You can specify which engine to use via the openssl.cnf configfile Or you can use the -engine parameter to specify an engine on the commandline.

From your C Code you can use ENGINE_by_id(engine_id);