According to JEP 131, Java 8 should provide a PKCS#11 Crypto provider for 64 bit Windows: https://blogs.oracle.com/mullan/entry/jep_131_pkcs_11_crypto.
With that in mind, I downloaded and built both 32 and 64 bit versions of NSS with NSPR using these instructions: https://developer.mozilla.org/en-US/docs/NSS_Sources_Building_Testing
I downloaded Java 8 for Windows 64 build b118, configured the java.security file and created a nss.cfg file:
Excerpt from java.security file:
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=sun.security.ec.SunEC
security.provider.4=com.sun.net.ssl.internal.ssl.Provider SunPKCS11-NSS
security.provider.5=com.sun.crypto.provider.SunJCE
security.provider.6=sun.security.jgss.SunProvider
security.provider.7=com.sun.security.sasl.Provider
security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.9=sun.security.smartcardio.SunPCSC
security.provider.10=sun.security.pkcs11.SunPKCS11 /devel/nss.cfg
nss.cfg:
# Use NSS as a FIPS-140 compliant cryptographic token
# SunPKCS11-NSS
name = NSS
#32 bit
nssLibraryDirectory = C:\devel\nss\nss-3.15.3.1\dist\WINNT6.1_DBG.OBJ\lib
#64 bit
#nssLibraryDirectory = C:\devel\nss\nss-3.15.3.1\dist\WINNT6.1_64_DBG.OBJ\lib
#non FIPS
#nssDbMode = noDb
#attributes = compatibility
#FIPS
nssSecmodDirectory = c:\devel\fipsdb
nssModule = fips
I ran the test suite that comes with NSS and it looks like all of the encryption/decryption tests passed (did have some issues with the tests that required hostname/domainname but that has to do with the Windows environment).
So here is the problem. I run my test encryption app on Java 7 32 bit with the 32 bit version of NSS and everything works great. When I attempt to run Java 8 64 bit with 64 bit NSS I get the following error:
java.security.ProviderException: Could not initialize NSS
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:212)
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:103)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at sun.security.jca.ProviderConfig$2.run(Unknown Source)
at sun.security.jca.ProviderConfig$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.jca.ProviderConfig.doLoadProvider(Unknown Source)
at sun.security.jca.ProviderConfig.getProvider(Unknown Source)
at sun.security.jca.ProviderList.getProvider(Unknown Source)
at sun.security.jca.ProviderList.getIndex(Unknown Source)
at sun.security.jca.ProviderList.getProviderConfig(Unknown Source)
at sun.security.jca.ProviderList.getProvider(Unknown Source)
at java.security.Security.getProvider(Unknown Source)
at sun.security.ssl.SunJSSE.<init>(Unknown Source)
at sun.security.ssl.SunJSSE.<init>(Unknown Source)
at com.sun.net.ssl.internal.ssl.Provider.<init>(Unknown Source)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at sun.security.jca.ProviderConfig$2.run(Unknown Source)
at sun.security.jca.ProviderConfig$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.jca.ProviderConfig.doLoadProvider(Unknown Source)
at sun.security.jca.ProviderConfig.getProvider(Unknown Source)
at sun.security.jca.ProviderList.getProvider(Unknown Source)
at sun.security.jca.ProviderList$ServiceList.tryGet(Unknown Source)
at sun.security.jca.ProviderList$ServiceList.access$200(Unknown Source)
at sun.security.jca.ProviderList$ServiceList$1.hasNext(Unknown Source)
at javax.crypto.KeyGenerator.nextSpi(KeyGenerator.java:323)
at javax.crypto.KeyGenerator.<init>(KeyGenerator.java:158)
at javax.crypto.KeyGenerator.getInstance(KeyGenerator.java:208)
at STSAESEncryption.generateKeyWithGenerator(STSAESEncryption.java:74)
at Main.main(Main.java:24)
Caused by: java.io.IOException: %1 is not a valid Win32 application.
at sun.security.pkcs11.Secmod.nssLoadLibrary(Native Method)
at sun.security.pkcs11.Secmod.initialize(Secmod.java:210)
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:207)
... 36 more
I did post a message to Sean Mullan's blog (linked above) and posted a reply to the question: everything is running 64 bit and I am unable to get it working in non-FIPS mode (same error) but my reply has not shown up on the blog yet (needs approval).
Has anyone else attempted to get NSS working with Java 8 64 bit on Windows 64 bit?
Update 1 based on Alex Pakka comment:
Thank you for the reply. I am using the 64bit NSS Library when I am using the Java 8 64 bit. Been switching back and forth as I test things both 32 and 64 bit.
I attached the code and stepped through but when I try to view the platformPath variable I get "platformPath cannot be resolved to a variable". I am not really that familiar with Eclipse so I am wondering if I am doing something wrong.
I have tried to edit the paths that I am putting in to see what errors I get and when I change the nssLibraryPath to another directory (without the nss library) I get a different error then the win32 one.
I do know that nss works with Java 8 64 bit for Linux (and possibly other platforms) but does has it been verified for Windows 64 bit. I know this is a new feature with Java 8 and Windows 64 bit with Java 7 only supporting Windows 43 bit.
Thanks again for the reply, it has helped and I am still trying to figure this out.
I would attach the source code e.g. from http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/sun/security/pkcs11/Secmod.java?av=f and put a breakpoint to check
platformPath
variable at line 210 (in the openjdk 7 code it is line 203). It does look like a PATH problem. NSS works with Java 8 64bit.The message "%1 is not a valid Win32 application" is misleading and comes from Windows itself, it could simply mean that nss3.dll was not found on library path.
Also, in your code you commented out in nss.cfg
You can only load 64 bit libraries into 64 bit Java process, so this path needs to be un-commented, and 32bit path should be commented out.