Exception while file signing using HSM and SUNPKCS11

Question

We are using the HSM(Hardware Security Module) for the signing and encryption from our java application. We have plugged the HSM to our JVM using the java.security property file using the following entry

security.provider.11=sun.security.pkcs11.SunPKCS11 D:/security/safenet.cfg

It was working fine all these days and suddenly we encountered a error as follows.

java.security.SignatureException: RSASignature::engineSign sun.security.pkcs11.P11Key$P11PrivateKey cannot be cast to java.security.interfaces.RSAPrivateKey

We did the enough searching on google but not able to identify the root cause.

Appreciate any help

Thanks

Answer 1

We also encountered the same issue. Specify the provider name when getting the Signing instance. When code to sign using HSM is called, there may some other piece of code in your application adding another provider [Security.insertProvider] which implements the same signing algorithm. When your sign, you pass a parameter of Private Key handler for PKCS11 but the provider is expecting different Key handler of RSA. Resolution: When you get the HSM signing instance, specify the Security provider along with the signing algorithm.