- 249 Views
- 1 replies
- 3 kudos
Not Every Pipeline Needs DLT — Here's How We Decided
Why We Used Delta Live Tables for Exactly One Pipeline (and Not for Silver/Gold)"Why isn't DLT used everywhere?" is one of the first questions I get when reviewing this architecture with other engineers — so I figured it was worth writing up properly...
- 249 Views
- 1 replies
- 3 kudos
- 3 kudos
I must admit, this was a nagging question I never asked as a junior data engineer and have just taken for granted every since. Thanks for addressing an oversight I didnt even know I had!
- 3 kudos
- 6119 Views
- 4 replies
- 9 kudos
Building a Metadata Table-Driven Framework Using LakeFlow Declarative (Formerly DLT) Pipelines
IntroductionScaling data pipelines across an organization can be challenging, particularly when data sources, requirements, and transformation rules are always changing. A metadata table-driven framework using LakeFlow Declarative (Formerly DLT) enab...
- 6119 Views
- 4 replies
- 9 kudos
- 9 kudos
can you please share the details how this can be implemented using a sample use case in step by step process. Also python code that needs to written in each layer (bronze/silver/gold)
- 9 kudos
- 2256 Views
- 4 replies
- 12 kudos
[Blog] Building a Scalable Telco CDR Processing Pipeline with Databricks Delta Live Tables - Part 1
Hey everyone! I wanted to share what I'm working with daily in the Databricks ecosystem and how amazing it is that we can achieve everything within one platform!Just published a deep dive on building a Telco CDR Processing Pipeline using: Delta Live ...
- 2256 Views
- 4 replies
- 12 kudos
- 12 kudos
@Pat here I was, cup of tea in hand, ready and eager to read the blog, only then did I discover I was redirected to another place that actually hosts the full blog .Personally, I think it'd be nice for it to be on here, especially if it's dedicated t...
- 12 kudos