Giters

kafka-ops / julie

A solution to help you build automation and gitops in your Apache Kafka deployments. The Kafka gitops!

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

jvm security options enforce use of truststore and keystore in 4.2.5, this a backwards incompatible change and diff behaviour than normal kafka logic

Fobhep opened this issue · comments

commented

Describe the bug
Julie does not accept trustore props via config file

To Reproduce
Julie-Properties File:

bootstrap.servers=broker1:9094
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-secret";
ssl.truststore.location=/home/atix/julie/kafka_broker.truststore.jks
ssl.truststore.password=confluenttruststorepass

results in multiple retries and fails finally:

shouldRetry: count=9
shouldRetry: count=10
shouldRetry: count=11
shouldRetry: count=12
shouldRetry: count=13
shouldRetry: count=14
shouldRetry: count=15
shouldRetry: count=16
shouldRetry: count=17
shouldRetry: count=18
shouldRetry: count=19
[ERROR] 2022-05-03 13:47:13.830 [main] MDSApiClient - java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[ERROR] 2022-05-03 13:47:13.830 [main] MDSApiClient - java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[ERROR] 2022-05-03 13:47:13.830 [main] MDSApiClient - java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
java.io.IOException: java.io.IOException: java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at com.purbon.kafka.topology.AccessControlProviderFactory.get(AccessControlProviderFactory.java:64)
	at com.purbon.kafka.topology.JulieOps.build(JulieOps.java:92)
	at com.purbon.kafka.topology.CommandLineInterface.processTopology(CommandLineInterface.java:211)
	at com.purbon.kafka.topology.CommandLineInterface.run(CommandLineInterface.java:161)
	at com.purbon.kafka.topology.CommandLineInterface.main(CommandLineInterface.java:147)
Caused by: java.io.IOException: java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at com.purbon.kafka.topology.api.mds.MDSApiClient.authenticate(MDSApiClient.java:106)
	at com.purbon.kafka.topology.AccessControlProviderFactory.get(AccessControlProviderFactory.java:57)
	... 4 more
Caused by: java.io.IOException: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at com.purbon.kafka.topology.clients.JulieHttpClient.doGet(JulieHttpClient.java:224)
	at com.purbon.kafka.topology.clients.JulieHttpClient.doGet(JulieHttpClient.java:162)
	at com.purbon.kafka.topology.api.mds.MDSApiClient.authenticate(MDSApiClient.java:95)
	... 5 more
Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
	at com.purbon.kafka.topology.clients.JulieHttpClient.doGet(JulieHttpClient.java:220)
	... 7 more
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:353)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:296)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:291)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.lambda$executeTasks$3(SSLFlowDelegate.java:1073)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:153)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.executeTasks(SSLFlowDelegate.java:1068)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.doHandshake(SSLFlowDelegate.java:1034)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:442)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:264)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
	... 21 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 27 more

Same properties file works with "normal" Admin-Clients like kafka-topics, kafka-console-consumer etc

Workaround:

 export JULIE_OPS_OPTIONS="-Djavax.net.ssl.trustStore=/home/atix/julie/kafka_broker.truststore.jks -Djavax.net.ssl.trustStorePassword=confluenttruststorepass"

prior to running works.

Expected behavior
Setting JVM Args via config file should work as well.

Runtime (please complete the following information):

  • OS: Rocky 8
  • JVM version: openjdk 11
  • Version 4.2.5
commented

Moin, this will not work in JulieOps, especially because of

julie/src/main/java/com/purbon/kafka/topology/clients/JulieHttpClient.java

Lines 93 to 100 in 231ca9e

if (areKeyStoreConfigured(config)) {
var keyManagers = getKeyManagersFromKeyStore(config);
var trustManagers = getTrustManagersFromTrustStore(config);
sslContext.init(keyManagers, trustManagers, null);
} else {
LOGGER.debug("Keystore and Trusstore not configured, connection will be using plain HTTP");
sslContext = SSLContext.getDefault();
}

in your example, kinda makes sense to pass only the trusstore as our authentication is done via the user/password. But taking into consideration that with TLS you're always passing a user cert, passing only the truststore could be questionable.

I understand why the difference in behaviour with Kafka could be annoying, but not sure I see it as an issue right now, I have a tendency to think this is "encouraging" better use of keystore ;-). No intention to be unpolite with your pain here, but looking forward to having a way how people see it in general.

Looking forward to elaborate on this.