https://community.databricks.com/t5/data-engineering/mqtt-to-delta-live-table/m-p/114101#M44721 <P>How this solution end up?</P> Mon, 31 Mar 2025 16:53:03 GMT KK0001 2025-03-31T16:53:03Z https://community.databricks.com/t5/data-engineering/mqtt-to-delta-live-table/m-p/37181#M26281 <P>Greetings,</P><P>I see that Delta Live Tables has various real-time connectors such as Kafka, Kinesis, Google's Pub Sub, and so on. I also see that Apache had maintained an mqtt connector to Spark through the 2.x series called <A href="https://bahir.apache.org/docs/spark/current/spark-streaming-mqtt/" target="_self">Bahir</A>, but dropped it in version 3.x which we are currently using.</P><P>My use case updates from each node once a minute a variable data payload from 18 KB to 100 KB 24/7/365. I am focusing on one node which averages 95 KB per minute. Log data can come faster than the 1 minute window. We use AWS and the AWS IoT mqtt.</P><P>How do I connect the mqtt to a Delta Live Table? Does it require an intermediary of Kafka/Kinesis?</P><P>As a intermediate solution, I have an IoT Rule to write an mqtt subscription directly to S3. I doubt that it will be performant but I do intend to test the Autoloader to DLT via S3 capability since it's right in front of me.</P><P>Cheers, Joe</P> Fri, 07 Jul 2023 18:21:43 GMT https://community.databricks.com/t5/data-engineering/mqtt-to-delta-live-table/m-p/37181#M26281 jhgorse 2023-07-07T18:21:43Z https://community.databricks.com/t5/data-engineering/mqtt-to-delta-live-table/m-p/114101#M44721 <P>How this solution end up?</P> Mon, 31 Mar 2025 16:53:03 GMT https://community.databricks.com/t5/data-engineering/mqtt-to-delta-live-table/m-p/114101#M44721 KK0001 2025-03-31T16:53:03Z