IntroductionIoT devices generate a large amount of data every day. In a perfect scenario, all this data arrives on time and our ingestion pipeline processes it using incremental loads. But in real-world IoT systems, this does not always happen.A devi...
Why I Built ThisAfter nearly 15 years working in enterprise supply chain and data engineering — most recently as a Staff Engineer at a large national grocery retailer — I have seen firsthand how retail organizations struggle with one persistent probl...
Really solid project—especially the focus on turning ML forecasts into practical order recommendations. The data quality and decision-engine approach makes it genuinely useful.
Part 5 - the final post in my enterprise data platform series - is up. This one is about what actually happened when we connected BI tools to the Views layer.We had two tools: Tableau for the finance team's analytical workbooks, and Databricks Dashbo...
Hi everyone,I recently published a technical blog on Z-Ordering vs Liquid Clustering in Delta Lake, covering the internals of both techniques in detail.Rather than focusing only on the syntax, the blog goes deeper into: The origin of Z-Ordering — tra...
IntroductionCloud-native data platforms like Azure Databricks are powerful because they abstract away infrastructure so you can focus on data engineering, analytics, and ML workloads. However, there are situations where you may run into issues that r...
