Resolved! Architecture choice, streaming data
I have sensor data coming into Azure Event Hub and need some help in deciding how to best ingest it into the Data Lake and Delta Lake:Option 1:azure event hub > databricks structured streaming > delta lake (bronze)Option 2:azure event hub > event hu...
- 6485 Views
- 4 replies
- 7 kudos
Latest Reply
If batch job is possible and you need to process data I would use probably:azure event hub from (events after previous job run) > databricks job process as dataframe > save df to delta lakeno streaming or capturing needed in that case
- 7 kudos