Kafka integration with Databricks

[Ajay-Pandey]

Hi all

I want to integrate Kafka with databricks if anyone can share any doc or code it will help me a lot.

Thanks in advance

[Hubert-Dudek]

This is code that I am using to read from Kafka

inputDF = (spark
.readStream
.format("kafka")
.option("kafka.bootstrap.servers", host)
.option("kafka.ssl.endpoint.identification.algorithm", "https")
.option("kafka.sasl.mechanism", "PLAIN")
.option("kafka.security.protocol", "SASL_SSL")
.option("kafka.sasl.jaas.config", "kafkashaded.org.apache.kafka.common.security.plain.PlainLoginModule required username='{}' password='{}';".format(userid, password))
.option("subscribe", topic)
.option("kafka.client.id", "Databricks")
.option("kafka.group.id", "new_group2")
.option("spark.streaming.kafka.maxRatePerPartition", "5")
.option("startingOffsets", "earliest")
.option("kafka.session.timeout.ms", "10000")
 .option("minPartitions", sc.DefaultParallelism)
.load() )

[Kumarpandey]

Thanks @Hubert Dudek​