topic Databricks <-> Kafka - SSL handshake failed in Data Engineering

Databricks <-> Kafka - SSL handshake failed

Jayanth746 — Thu, 17 Nov 2022 17:36:53 GMT

I am receiving SSL handshake error even though the trust-store I have created is based on server certificate and the fingerprint in the certificate matches the trust-store fingerprint.

kafkashaded.org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed Caused by: java.security.cert.CertPathValidatorException: signature check failed Caused by: java.security.SignatureException: Signature does not match.

df = spark.readStream \

.format("kafka") \

.option("kafka.bootstrap.servers","XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX") \

.option("kafka.security.protocol", "SSL") \

.option("kafka.ssl.truststore.location",'/dbfs/dbfs/FileStore/Certs/client.truststore1der.jks' )

.option("kafka.ssl.keystore.location", '/dbfs/dbfs/FileStore/Certs/client.keystore.jks') \

.option("kafka.ssl.keystore.password", keystore_pass) \

.option("kafka.ssl.truststore.password", truststore_pass) \

.option("kafka.ssl.keystore.type", "JKS") \

.option("kafka.ssl.truststore.type", "JKS") \

.option("kafka.ssl.endpoint.identification.algorithm","") \

.option("subscribe","bets") \

.load()

Re: Databricks <-> Kafka - SSL handshake failed

Jayanth746 — Fri, 18 Nov 2022 09:09:54 GMT

Hi @Debayan Mukherjee , I have already specified the option .option("kafka.ssl.endpoint.identification.algorithm","") \.

My error is specific to signature not matching

java.security.SignatureException: Signature does not match.

The 2nd link is not working

Re: Databricks <-> Kafka - SSL handshake failed

Debayan — Fri, 18 Nov 2022 07:18:44 GMT

Hi @Jayanth Goulla , worth a try ,

https://stackoverflow.com/questions/54903381/kafka-failed-authentication-due-to-ssl-handshake-failed

Did you follow: https://docs.microsoft.com/en-us/azure/databricks/spark/latest/structured-streaming/kafka?