Databricks

mj2022 · 06-30-2022

I testing Spark Streaming working withSASL_SSL enabled kafka broker in a notebook.as per this guide https://docs.databricks.com/spark/latest/structured-streaming/kafka.htmli have copied jsk files in an s3 bucket and mounted it in dbfs.In notebook wh...

mj2022 · 07-27-2022

I following the https://docs.databricks.com/dev-tools/databricks-connect.html#intellij-scala-or-java to obtain spark-avro jar since databricks have it's custom from_avro method to use with kafka schema registry, But i am not able to find spark-avro j...

mj2022 · 07-26-2022

Thanks..Yes '/dbfs/mnt/xxxx/kafka.client.truststore.imported.jks' path worked. Also other workaround we got it working, is copy the file from s3 to filesystem using init script and use filepath.

mj2022 · 06-30-2022

I am getting similar error when i connect to kafka using spark streaming from notebook or using spark job.I can find the jks file in mounted dbfs location, but spark streaming complainsCaused by: kafkashaded.org.apache.kafka.common.KafkaException: ka...

Databricks

User Stats

User Activity

Spark Streaming with SASL_SSL Kafka throwing java.nio.file.NoSuchFileException: dbfs:/mnt/**/kafka.client.truststore.imported.jks

Re: Does Databricks have a maven repository to download the jars?

Re: Spark Streaming with SASL_SSL Kafka throwing java.nio.file.NoSuchFileException: dbfs:/mnt/**/kafka.client.truststore.imported.jks

Re: Connecting Databricks to Heroku Kafka