- 474 Views
- 2 replies
- 4 kudos
INSERT WITH SCHEMA EVOLUTION
I am back, and runtime 18.1 is here, and with it INSERT WITH SCHEMA EVOLUTION https://databrickster.medium.com/databricks-news-2026-week-8-16-february-2026-to-22-february-2026-f2ec48bc234f
- 474 Views
- 2 replies
- 4 kudos
- 548 Views
- 2 replies
- 2 kudos
The Mythical Man-Month Revisited: Engineering in the Era of AI Agents
For more than 50 years, software engineering has been guided by principles like The Mythical Man-Month, Brooks’s Law, Hofstadter’s Law, Parkinson’s Law, and the Iron Triangle.But those laws were built around a specific reality: human cognitive limits...
- 548 Views
- 2 replies
- 2 kudos
- 2 kudos
Love this post, Sudhir.It's been fascinating to see how quickly the principles in TMMM are evolving in the last 6 months, with the rise of agentic AI. It's a shame Fred Brooks is not around to see this technology shift.
- 2 kudos
- 204 Views
- 0 replies
- 3 kudos
Foundation for Agentic Quality Monitoring
Agentic quality monitoring is available in databricks. But tooling alone is not enough. You need a clearly defined Data Quality Pillar across your Lakehouse architecture. #databricks https://www.sunnydata.ai/blog/databricks-data-quality-pillar-ai-rea...
- 204 Views
- 0 replies
- 3 kudos
- 226 Views
- 0 replies
- 1 kudos
Data deduplication
At the Lakehouse, we don't enforce Primary Keys, which is why the deduplication strategy is so important. One of my favourites is using transformWithStateInPandas. Of course, it only makes sense in certain scenarios. See all five major strategies on ...
- 226 Views
- 0 replies
- 1 kudos
- 279 Views
- 0 replies
- 1 kudos
Databricks is event-driven
When a new external location is created, file events are created by default. The autoloader from runtime 18.1 defaults to file notification mode.It is just a few weeks after TRIGGER ON UPDATE was introduced. more news:https://databrickster.medium.com...
- 279 Views
- 0 replies
- 1 kudos
- 465 Views
- 0 replies
- 3 kudos
🚀 Databricks Runtime 18.1 (Beta) — A Big Leap Forward for Data & AI Teams
Databricks has rolled out Runtime 18.1 (Beta), and it’s packed with meaningful enhancements across streaming, Delta Lake, SQL, geospatial, performance, and Spark 4.1.0 improvements. This release builds on 18.0 and introduces new capabilities that mak...
- 465 Views
- 0 replies
- 3 kudos
- 1526 Views
- 0 replies
- 0 kudos
State of AI Agents 2026: Lessons on Governance, Evaluations, and Scale
The Databricks State of AI Agents 2026 report highlights a clear reality: most AI pilots still fail to reach production, and governance and evaluation are the true differentiators.In my latest blog, I break down the key findings and share how we appr...
- 1526 Views
- 0 replies
- 0 kudos
- 237 Views
- 0 replies
- 1 kudos
Databricks Data + AI Summit 2026 meme — who do you recognize?
Registration for Databricks Data + AI Summit 2026 is open, so I made a quick meme video with recognizable Databricks/community faces.Who did you recognize? Who’s missing?Bonus: share one fun fact about any of them — I’ll compile the best ones for a f...
- 237 Views
- 0 replies
- 1 kudos
- 171 Views
- 0 replies
- 1 kudos
Materialized View Smart Feature
One amazing feature that really stands out for me after creating a Materialized View in Databricks is the Definition SQL panel under the Overview tab that shows the exact query behind the MV. It sounds simple, but it’s incredibly powerful.Being able ...
- 171 Views
- 0 replies
- 1 kudos
- 186 Views
- 0 replies
- 1 kudos
Stop Guessing Your Data’s Trust Level: Use Databricks System Tags
Recently, Databricks introduced System Tags, allowing you to explicitly label objects as Certified, Deprecated or None.Certification status system tagThe certified status system tag allows users to label objects, such as catalogs, schemas, tables, da...
- 186 Views
- 0 replies
- 1 kudos
- 239 Views
- 0 replies
- 2 kudos
Lakebase: a strategic business view
Lakebase is not just a product release, it’s a strategic architectural signal.As AI systems become agent-driven, ephemeral, and programmatic, the bottleneck is no longer analytics, it’s operational databases designed for permanence. Lakebase represe...
- 239 Views
- 0 replies
- 2 kudos
- 160 Views
- 0 replies
- 2 kudos
Publish to Power BI Online from Azure Databricks
In this video, I will show you how to seamlessly publish your Azure Databricks data directly to Power BI Online, eliminating the need for complex export-import workflows.You'll learn how to configure the necessary permissions, set up Unity Catalog i...
- 160 Views
- 0 replies
- 2 kudos
- 134 Views
- 0 replies
- 1 kudos
Google Sheets Pivots
Install databricks extension in Google Sheets, now it has a new cool functionality which allows generating pivots connected to UC data #databricks https://databrickster.medium.com/databricks-news-2026-week-6-2-february-2026-to-8-february-2026-1ae1630...
- 134 Views
- 0 replies
- 1 kudos
- 188 Views
- 0 replies
- 1 kudos
Agile and Databricks
I originally wanted to publish this as a LinkedIn article.But as many of you know, the LinkedIn engine doesn’t really promote long-form articles anymore, so I turned it into a post instead.As we mark 25 years of the Agile Manifesto, I’ve been reflect...
- 188 Views
- 0 replies
- 1 kudos
- 156 Views
- 0 replies
- 2 kudos
Low-code LLM judges
MlFlow 3.9 introduces low-code, easy-to-implement LLM judges #databricks https://databrickster.medium.com/databricks-news-2026-week-6-2-february-2026-to-8-february-2026-1ae163015764
- 156 Views
- 0 replies
- 2 kudos
