Is it possible to configure Spring Cloud Netflix Eureka Server over mTLS. I have a Eureka Server in which I have enable mTLS ( I have set server.ssl.clientAuth=need ) and provided my custom CA as truststore.
For my Eureka clients applications I am able to register my application to Eureka Server over mTLS without any issue. And for DiscoverClient is my EurekaServer I have configured Jersey3DiscoveryClientOptionalArgs and so I am able to register my Eureka Server as a Eureka Client mTLS.
I am facing issues while trying to while Replicating to Peer Nodes over mTLS via Jersey3ReplicationClient . Initially I was getting "PKIX path building failed" error. For Which I have update my JVM cacerts truststore.
Now I am getting "bad_certificate"
Below is the SSL Handshake debug log. I have only included the relevant part.
I can see while Replicating to Peer Nodes.
Server Certificate is produced, Which then trusted by the Jersey3ReplicationClient ( Because I added the issuing CA Certificate to JVM Truststore in JAVA HOME ) . Then I can see the Jersey3ReplicationClient is not producing any certificate/key to authenticate it self (Empty client certificate chain). As mTLS is enabled, Replication to Peer Nodes is failing.
javax.net.ssl|DEBUG|04|https-jsse-nio-9080-exec-4|2024-02-03 06:32:42.014 IST|CertificateMessage.java:1172|Consuming client Certificate handshake message (
"Certificate": {
"certificate_request_context": "",
"certificate_list": [
]
}
)
javax.net.ssl|DEBUG|F2|TaskBatchingWorker-target_localhost-11|2024-02-03 06:32:42.015 IST|SSLCipher.java:2024|KeyLimit write side: algorithm = AES/GCM/NOPADDING:KEYUPDATE
countdown value = 137438953472
javax.net.ssl|ERROR|04|https-jsse-nio-9080-exec-4|2024-02-03 06:32:42.018 IST|TransportContext.java:370|Fatal (BAD_CERTIFICATE): Empty client certificate chain (
"throwable" : {
javax.net.ssl.SSLHandshakeException: Empty client certificate chain
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:312)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1188)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1277)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1264)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1209)
at org.apache.tomcat.util.net.SecureNioChannel.tasks(SecureNioChannel.java:427)
at org.apache.tomcat.util.net.SecureNioChannel.handshakeUnwrap(SecureNioChannel.java:492)
at org.apache.tomcat.util.net.SecureNioChannel.handshake(SecureNioChannel.java:213)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1719)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:842)}
javax.net.ssl|DEBUG|F2|TaskBatchingWorker-target_localhost-11|2024-02-03 06:32:42.029 IST|SSLSocketImpl.java:1755|close the underlying socket
javax.net.ssl|DEBUG|F2|TaskBatchingWorker-target_localhost-11|2024-02-03 06:32:42.029 IST|SSLSocketImpl.java:1781|close the SSL connection (initiative)
javax.net.ssl|WARNING|F2|TaskBatchingWorker-target_localhost-11|2024-02-03 06:32:42.029 IST|SSLSocketImpl.java:1672|handling exception (
"throwable" : {
javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1509)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1480)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1065)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:483)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:662)
at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:697)
at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:691)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:691)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:661)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:439)
at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:345)
at com.netflix.eureka.transport.Jersey3ReplicationClient.submitBatchUpdates(Jersey3ReplicationClient.java:116)
at com.netflix.eureka.cluster.ReplicationTaskProcessor.process(ReplicationTaskProcessor.java:80)
at com.netflix.eureka.util.batcher.TaskExecutors$BatchWorkerRunnable.run(TaskExecutors.java:190)
at java.base/java.lang.Thread.run(Thread.java:842)}
Is there anyway to configure trustore and keystore for the Replication Client like I have configured for the Jersey Discovery Client via Jersey3DiscoveryClientOptionalArgs.
My main objectives is to enable mTLS is Eureka Server cluster with Replication enabled, But the Replication Client is not producing any Client Certificate.
I tried to find to update SSLContext for this Jersey Replication Client, but seems there is no straight forward way to do so.
I have also tried using VM arguments like -Djavax.net.ssl.keyStore but still the Client Certificate produced by the Jersey Replication Client is empty.
How to achieve this?
My pom.xml
<properties>
<java.version>17</java.version>
<spring-cloud.version>2023.0.0</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
My application.yml file
spring:
application:
name: EUREKA-SERVER
server:
port: 9080
ssl:
enabled: true
protocol: TLS
keyStore: classpath:keystore/eureka_server_mTLS.p12
keyStorePassword: mypassword
keyStoreType: PKCS12
keyAlias: eurekaserver
clientAuth: need
trustStore: classpath:truststore/CA_mTLS_Soumyadip.jks
trustStorePassword: mypassword
trustStoryType: JKS
http:
enabled: false
eureka:
instance:
nonSecurePortEnabled: false
securePortEnabled: true
securePort: ${server.port}
client:
registerWithEureka: true
fetchRegistry: false
tls:
trustStore: classpath:truststore/CA_mTLS_Soumyadip.jks
trustStorePassword: mypassword
trustStoreType: JKS
keyStore: classpath:keystore/eureka_server_mTLS.p12
keyPassword: mypassword
keyStorePassword: mypassword
keyAlias: eurekaserver
enabled: true
keyStoreType: PKCS12
serviceUrl:
defaultZone: https://192.168.29.6:9081/eureka/