For years, enterprise data architecture has followed a familiar pattern.An application writes customer orders, account updates, inventory changes, or transactions into an operational database.Then data engineering takes over.We capture changes throug...
IntroductionDifferent organizations use different names for these artifacts: business requirements, mapping specifications, source-to-target mappings, data contracts, transformation rules, or semantic definitions. The name matters less than the goal:...
Overview Prompted by a customer question, I wanted to see what was possible in terms of MCP integration into Genie Code, in order to try this out I decided to look at Azure Dev Ops, as it's a common workflow to want to see your tickets alongside the ...
Hi Emma how are you? is there any solution to make this agent avaliable for multiple users? the main restriction is that the path will be linked to a single user/service princpal.thanks!
A question came up in the community recently that I thought deserved more than a short answer. The question was around how to build a reliable investigation sequence for slow Spark jobs, specifically when symptoms overlap. A long-running stage with ...
Thanks for this! Very insightful and detailed. Your sequence for diagnosing slow Spark jobs when symptoms overlap is exactly what I needed. Bookmarked this for my team.
One Question Can Light the SparkMy Databricks journey started in 2022 with simple interest, curiosity, and a dream to learn more.At that time, I was just trying to understand the platform, follow the updates, learn from others, and slowly build my co...
My turn! (couldn't let this disappear from the homepage that quickly ).This one is officially going down in Community history now, @Brahmareddy .
If you've ever worked on an insurance data warehouse, or really any warehouse where data arrives from different systems at different times, you know the pain of keeping things in sync. I spent years building data warehouses for a property and casual...
I know I'm reading this a bit late, but it's absolutely worth the read. Excellent write-up!Please share link for part-2
Stop Writing ETL Code for Every New PipelineAs organizations modernize their data platforms, one challenge continues to appear repeatedly:Every new source requires another ETL job.Every new business rule requires another SQL update.Every schema chan...
IntroductionApache Spark has become one of the most widely adopted engines for large-scale data processing. Its appeal is easy to understand: it supports batch processing, streaming workloads, feature engineering, machine learning pipelines, and lar...
Just back from Data and AI Summit 2026 and I am still buzzing.This was my best summit yet. Moscone was packed. More than 30,000 of us in one place, from over 150 countries, all there for the same reason. To build better with data and AI.The keynotes ...
@Brahmareddy Thank you so much for your support of the Community! It was such a delight to see you at the Community booth and meet up in person. I'm looking forward to working with you as a Community Champion and making this an even better place for ...
Databricks CustomerLake: Inside the Agentic CDP Built for the Age of AIA deep dive into what CustomerLake actually is, how it works, and what it looks like in practice.At Data + AI Summit 2026, Databricks announced CustomerLake — an Agentic Customer ...
Part 4 of my enterprise data platform series is up - this one cover why we added a fifth layer to the standard medallion architecture.We connected BI tools to the Gold layer and immediately hit four problems Gold alone couldn't solve:Schema breaks wh...
Hello Databricks Community!I recently published a detailed breakdown on Medium about a real-world optimization nightmare we faced, and I wanted to share the core lessons learned with this group.We had a highly efficient Delta table pipeline handling ...
Delta Lake gives you time travel backward:SELECT * FROM sales TIMESTAMP AS OF '2026-01-01'But what about forward? What will your Delta table look like next month?Nobody has built probabilistic future queries as a first-class Delta concept — until now...
The most important thing I took away from Data + AI Summit 2026 was not one product announcement.It was the direction.Databricks is building around a very real enterprise problem: companies want AI to help with decisions, operations, customer engagem...
Hi folks, So I have recently, as a side project been working on a tool, Link to my github: https://github.com/JeffDenzel/lineage-syncer.The problemThe reason for it is that DataBricks allows you to put external data assets into your UC. However, ther...
This is great work, Jeff!I have had my own share of pain with Power BI reports breaking because of column/name changes, so this problem feels very real.Linking Power BI semantic models back to Unity Catalog lineage sounds like a very useful directio...
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
