I recently published a piece on Lakeflow Connect and wanted to share it here since this community is where the conversation actually happens.The post covers something most of us have lived through, the hidden cost of maintaining ingestion pipelines. ...
Hello Guys,I am planning to implement Event Driven Data Ingestion from Bronze -> Silver -> Gold layer in my project. Currently we are having batch processing approach for our data ingestion pipelines. We have decided to move away from batch process t...
Hi Mey,Please also consider databrick file arrival trigger for your event driven data ingestion journey.https://docs.databricks.com/aws/en/jobs/file-arrival-triggersRegards, Kartik
