TLS 1.2 (RSA mode: ex AES256-GCM-SHA384 ) does't work on red hat 9 (openssl 3.0.7-16.el9_2 ) with libp11 (version 0.4.7 and 0.4.8 and 0.4.11 and 0.4.12). I use an eToken (5110) to store the private key. The server is a stunnel with openssl 3.0 and the client is only openssl (openssl s_client -cert client.pem -key client.key -connect localhost:7001 -tls1_2 )
I got a padding error :
2023.07.10 14:21:55 LOG5[0]: Service [sa_T2_V3_test] accepted connection from 127.0.0.1:43658
2023.07.10 14:21:55 LOG6[0]: Peer certificate required
2023.07.10 14:21:55 LOG6[0]: Certificate accepted at depth=1: C=XX, L=Valence, O=XXXXX, OU=BU, CN=ROOT CA SERVER/TPE
2023.07.10 14:21:55 LOG6[0]: CERT: No subject checks configured
2023.07.10 14:21:55 LOG5[0]: Certificate accepted at depth=0: C=FR, ST=DR, L=Valence, O=XXXXX, OU=BU, CN=YYYYY
2023.07.10 14:21:55 LOG3[0]: error queue: ssl/statem/statem_srvr.c:2911: error:0A000093:SSL routines::decryption failed
2023.07.10 14:21:55 LOG3[0]: error queue: crypto/evp/pmeth_lib.c:1336: error:03000093:digital envelope routines::command not supported
2023.07.10 14:21:55 LOG3[0]: SSL_accept: crypto/rsa/rsa_pmeth.c:456: error:02000090:rsa routines::illegal or unsupported padding mode
See the last sentence in log: error:02000090:rsa routines::illegal or unsupported padding mode
The wireshark trace show an issue with the first RSA block to decrypt by the server.
It works on red hat 7 with a old openssl version. It works on openssl 3.0 on ECDHE (with no rsa).
I you have an idea...
thanks in advance.
a solution or a track to look for