- 482 Views
- 1 replies
- 1 kudos
DABS: external locations
More under DABS! External locations are now available as DABS code. I hope that credentials will be available soon, too, so it will be possible to reference the credential resource from an external location. #databricks https://medium.com/@databricks...
- 482 Views
- 1 replies
- 1 kudos
- 1 kudos
This is great, so eventually DAB might help replacing Terraform overhead for these platform resources or any other infrastructure maintenance and governance.
- 1 kudos
- 608 Views
- 0 replies
- 1 kudos
Migration from Azure Data Factory(ADF) to Databricks Lakeflow Jobs
Many teams think migrating from Azure Data Factory to Databricks Lakeflow Jobs is just a pipeline rewrite. It’s not.It’s about:* Simplifying over-engineered architectures* Reducing orchestration and monitoring overhead* Aligning data engineering with...
- 608 Views
- 0 replies
- 1 kudos
- 719 Views
- 2 replies
- 3 kudos
Catalogs in DABS
Catalogs are now under DABS, and I am happy to say goodbye to Terraform and to manage all UC grants in DABS. #databricks https://databrickster.medium.com/databricks-news-2026-week-8-16-february-2026-to-22-february-2026-f2ec48bc234f
- 719 Views
- 2 replies
- 3 kudos
- 805 Views
- 0 replies
- 5 kudos
Setting 1-Minute SQL Warehouse Auto-termination via API in Databricks
Hi everyone, I wanted to share a quick technical tip for those looking to optimize their SQL Warehouse configurations, especially when dealing with intermittent ad hoc queries or AI BI Dashboards. The Challenge: When setting up a SQL Warehouse, the D...
- 805 Views
- 0 replies
- 5 kudos
- 241 Views
- 0 replies
- 2 kudos
set query tags
It is possible to tag queries. That functionality is also supported by external clients (Jdbc/dbt/Power BI, etc.) #databricks https://databrickster.medium.com/databricks-news-2026-week-8-16-february-2026-to-22-february-2026-f2ec48bc234f
- 241 Views
- 0 replies
- 2 kudos
- 295 Views
- 0 replies
- 3 kudos
just TABLE
Do you know that instead of SELECT * FROM TABLE, you can just use TABLE? TABLE is just part of pipe syntax, so you can always add another part after the pipe. Thanks to Martin Debus for noticing the possibility of using just TABLE. #databricks https:...
- 295 Views
- 0 replies
- 3 kudos
- 665 Views
- 2 replies
- 4 kudos
INSERT WITH SCHEMA EVOLUTION
I am back, and runtime 18.1 is here, and with it INSERT WITH SCHEMA EVOLUTION https://databrickster.medium.com/databricks-news-2026-week-8-16-february-2026-to-22-february-2026-f2ec48bc234f
- 665 Views
- 2 replies
- 4 kudos
- 928 Views
- 2 replies
- 2 kudos
The Mythical Man-Month Revisited: Engineering in the Era of AI Agents
For more than 50 years, software engineering has been guided by principles like The Mythical Man-Month, Brooks’s Law, Hofstadter’s Law, Parkinson’s Law, and the Iron Triangle.But those laws were built around a specific reality: human cognitive limits...
- 928 Views
- 2 replies
- 2 kudos
- 2 kudos
Love this post, Sudhir.It's been fascinating to see how quickly the principles in TMMM are evolving in the last 6 months, with the rise of agentic AI. It's a shame Fred Brooks is not around to see this technology shift.
- 2 kudos
- 269 Views
- 0 replies
- 3 kudos
Foundation for Agentic Quality Monitoring
Agentic quality monitoring is available in databricks. But tooling alone is not enough. You need a clearly defined Data Quality Pillar across your Lakehouse architecture. #databricks https://www.sunnydata.ai/blog/databricks-data-quality-pillar-ai-rea...
- 269 Views
- 0 replies
- 3 kudos
- 296 Views
- 0 replies
- 1 kudos
Data deduplication
At the Lakehouse, we don't enforce Primary Keys, which is why the deduplication strategy is so important. One of my favourites is using transformWithStateInPandas. Of course, it only makes sense in certain scenarios. See all five major strategies on ...
- 296 Views
- 0 replies
- 1 kudos
- 369 Views
- 0 replies
- 1 kudos
Databricks is event-driven
When a new external location is created, file events are created by default. The autoloader from runtime 18.1 defaults to file notification mode.It is just a few weeks after TRIGGER ON UPDATE was introduced. more news:https://databrickster.medium.com...
- 369 Views
- 0 replies
- 1 kudos
- 675 Views
- 0 replies
- 3 kudos
🚀 Databricks Runtime 18.1 (Beta) — A Big Leap Forward for Data & AI Teams
Databricks has rolled out Runtime 18.1 (Beta), and it’s packed with meaningful enhancements across streaming, Delta Lake, SQL, geospatial, performance, and Spark 4.1.0 improvements. This release builds on 18.0 and introduces new capabilities that mak...
- 675 Views
- 0 replies
- 3 kudos
- 2606 Views
- 0 replies
- 0 kudos
State of AI Agents 2026: Lessons on Governance, Evaluations, and Scale
The Databricks State of AI Agents 2026 report highlights a clear reality: most AI pilots still fail to reach production, and governance and evaluation are the true differentiators.In my latest blog, I break down the key findings and share how we appr...
- 2606 Views
- 0 replies
- 0 kudos
- 388 Views
- 0 replies
- 1 kudos
Databricks Data + AI Summit 2026 meme — who do you recognize?
Registration for Databricks Data + AI Summit 2026 is open, so I made a quick meme video with recognizable Databricks/community faces.Who did you recognize? Who’s missing?Bonus: share one fun fact about any of them — I’ll compile the best ones for a f...
- 388 Views
- 0 replies
- 1 kudos
- 249 Views
- 0 replies
- 1 kudos
Materialized View Smart Feature
One amazing feature that really stands out for me after creating a Materialized View in Databricks is the Definition SQL panel under the Overview tab that shows the exact query behind the MV. It sounds simple, but it’s incredibly powerful.Being able ...
- 249 Views
- 0 replies
- 1 kudos
