https://community.databricks.com/t5/get-started-discussions/highly-performant-data-ingestion-and-processing-pipelines/m-p/54019#M6178 <P>Hi everyone,</P><P>I am working on a project that requires highly performant pipelines for managing data ingestion, validation, and processing large data volumes from IOT devices.<BR /><BR />I am interested in knowing:<BR />- The best way to ingest from EventHub/Kafka sinks<BR />- Data validation<BR />- Post-processing after data ingestion<BR />- Reprocessing incorrect data</P><P>If you have experience in this area and would be willing to chat with me, I would greatly appreciate it. I would like to know how you handle specific challenges and learn about your experience and best practices.</P><P>Please feel free to contact me to schedule a time to talk. I look forward to hearing from you.</P><P>Best regards,</P><P>Hector A Garcia</P> Mon, 27 Nov 2023 19:18:00 GMT hagarciaj 2023-11-27T19:18:00Z https://community.databricks.com/t5/get-started-discussions/highly-performant-data-ingestion-and-processing-pipelines/m-p/54019#M6178 <P>Hi everyone,</P><P>I am working on a project that requires highly performant pipelines for managing data ingestion, validation, and processing large data volumes from IOT devices.<BR /><BR />I am interested in knowing:<BR />- The best way to ingest from EventHub/Kafka sinks<BR />- Data validation<BR />- Post-processing after data ingestion<BR />- Reprocessing incorrect data</P><P>If you have experience in this area and would be willing to chat with me, I would greatly appreciate it. I would like to know how you handle specific challenges and learn about your experience and best practices.</P><P>Please feel free to contact me to schedule a time to talk. I look forward to hearing from you.</P><P>Best regards,</P><P>Hector A Garcia</P> Mon, 27 Nov 2023 19:18:00 GMT https://community.databricks.com/t5/get-started-discussions/highly-performant-data-ingestion-and-processing-pipelines/m-p/54019#M6178 hagarciaj 2023-11-27T19:18:00Z