A practical guide from 12 years of building Azure data platforms.If you've spent any time working on Azure data platforms, you've probably hit this question at some point — should I build this in Azure Data Factory or just do it in Databricks?I've be...
Hi everyone,I recently took a look into a silent cost driver in many data platforms: the default choice between managed and external tables in Unity Catalog.It is very common for teams to default to external tables, but this choice often leads to acc...
Hi Databricks support, I am looking for a standardized Databricks framework to update job definition using DevOps from non-production till it get productionized. Our current process of updating the Databricks job definition is as follows:In our sourc...
Hi from the Git folders/Repos PM: DAB is the way to go, and we are working on an integration to author DABs directly in the workspace. Here's a DAIS talk where the DAB PM and I demo'ed some recommendations for source controlling jobs: https://www.da...
