https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13578#M8236 <P>Hi,</P><P>@Hubert Dudek​&nbsp; can you give me some reference material or video to take help? </P> Thu, 05 Jan 2023 16:11:41 GMT qasimhassan 2023-01-05T16:11:41Z https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13576#M8234 <P>I'm working on the case to configure Kafka that is installed on my machine (Laptop) &amp; I want to connect it with my Databricks account hosted on the AWS cloud.</P><P></P><P>Secondly, I have CSV files that I want to use for real-time processing from Kafka to Databrick dbfs, where I'll process the data coming live from Kafka.</P><P></P> Thu, 05 Jan 2023 07:55:28 GMT https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13576#M8234 qasimhassan 2023-01-05T07:55:28Z https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13577#M8235 <P>For CSV, you need just to readStream in the notebook and append output to CSV using forEachBatch method.</P><P></P><P>Your Kafka on PC needs to have the public address or you need to set AWS VPN and connect from your laptop to be in the same VPC as databricks.</P> Thu, 05 Jan 2023 10:20:06 GMT https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13577#M8235 Hubert-Dudek 2023-01-05T10:20:06Z https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13578#M8236 <P>Hi,</P><P>@Hubert Dudek​&nbsp; can you give me some reference material or video to take help? </P> Thu, 05 Jan 2023 16:11:41 GMT https://community.databricks.com/t5/data-engineering/how-to-kafka-configured-on-your-pc-with-databricks/m-p/13578#M8236 qasimhassan 2023-01-05T16:11:41Z