Error Reading Kafka message into Azure Databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-08-2024 01:34 PM
Team
I am trying to test the connection to Kafka broker from Azure Databricks. Telnet and IP is successful.
When I am trying to read the data, I am getting "Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target" error.
For reference below is the code:
connString ="kafkashaded.org.apache.kafka.common.security.plain.PlainLoginModule required username=\"uid\" password=\"pwd\";"
df = (spark
.readStream
.format("kafka")
.option("subscribe", "NOTIFICATIONS")
.option("kafka.bootstrap.servers", "brkip:9093")
.option("kafka.security.protocol", "SASL_SSL")
.option("kafka.sasl.mechanism", "PLAIN")
.option("kafka.sasl.jaas.config", connString)
.option("kafka.request.timeout.ms", "60000")
.option("kafka.session.timeout.ms", "30000")
.load())
display(df)
Am I missing any thing? Any suggestions?
Regards
Sree