- 464 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #8
Data classification automatically tags Unity Catalog tables and is now available in system tables as well.
- 464 Views
- 0 replies
- 1 kudos
Forum Posts
- 2436 Views
- 4 replies
- 5 kudos
Spark Jobs View on a serverless cluster!! Naah.....
Have you ever noticed (and wondered) that the wonderful Spark Job UI is no longer available in the databricks notebook if the cell is executed using 'serverless' cluster?Tradionally, whenever we run the spark code (action command), we used to see the...
- 2436 Views
- 4 replies
- 5 kudos
Latest Reply
- 5 kudos
Hi RamanThank you for the amazing insights! I am trying to understand more about SQL Warehouses - is it managed by Unity Catalog? From what I could gather, SQL Warehouse is a compute layer, not a data layer and therefore not managed by Unity Catalog....
- 5 kudos
- 506 Views
- 0 replies
- 2 kudos
Databricks Advent Calendar 2025 #7
Imagine all a data engineer or analyst needs to do to read from a REST API is use spark.read(), no direct request calls, no manual JSON parsing - just spark .read. That’s the power of a custom Spark Data Source. Soon, we will see a surge of open-sour...
- 506 Views
- 0 replies
- 2 kudos
- 533 Views
- 0 replies
- 2 kudos
Databricks Advent Calendar 2025 #6
DBX is one of the most crucial projects of dblabs this year, and we can expect that more and more great checks from it will be supported natively in databricks. More about dbx on https://databrickslabs.github.io/dqx/
- 533 Views
- 0 replies
- 2 kudos
- 623 Views
- 1 replies
- 2 kudos
Databricks Advent Calendar 2025 #5
When something goes wrong, and your pattern is doing MERGEs per day in your jobs, backfill jobs will help you to reload many days in one shot.
- 623 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Full link of the actual blog for reference - https://www.databricks.com/blog/announcing-backfill-runs-lakeflow-jobs-higher-quality-downstream-data
- 2 kudos
- 847 Views
- 1 replies
- 4 kudos
Databricks Advent Calendar 2025
With the first day of December comes the first window of our Databricks Advent Calendar. It’s a perfect time to look back at this year’s biggest achievements and surprises — and to dream about the new “presents” the platform may bring us next year. ...
- 847 Views
- 1 replies
- 4 kudos
Latest Reply
- 4 kudos
Fantastic kickoff to the Databricks Advent Calendar 2025 , appreciate you steering the series, @Hubert-Dudek!
- 4 kudos
- 498 Views
- 0 replies
- 4 kudos
Databricks Advent Calendar 2025 #4
With the new ALTER SET, it is really easy to migrate (copy/move) tables. Quite awesome also when you need to make an initial load and have an old system under Lakehouse Federation (foreign tables).
- 498 Views
- 0 replies
- 4 kudos
- 6317 Views
- 0 replies
- 0 kudos
Building a Claude Code-Compatible Proxy for Databricks (with MCP, Git Tools, and Prompt Caching)
Many Databricks engineers have asked whether it's possible to use Claude Code CLI directly against Databricks-hosted Claude models instead of Anthropic's cloud API. This enables repo-aware AI workflows—navigation, diffs, testing, MCP tools—right insi...
- 6317 Views
- 0 replies
- 0 kudos
- 843 Views
- 2 replies
- 5 kudos
Databricks Advent Calendar 2025 #3
One of the biggest gifts is that we can finally move Genie to other environments by using the API. I hope DABS comes soon.
- 843 Views
- 2 replies
- 5 kudos
Latest Reply
- 387 Views
- 0 replies
- 0 kudos
Fast-Track Your Databricks Development: Using Widgets for Test and Production Modes
Test Mode Pattern with Databricks Widgets - Demo What's IncludedThis package contains comprehensive documentation and examples for implementing the Test Mode Pattern in Databricks notebooks using widgets. This pattern enables fast development cycles ...
- 387 Views
- 0 replies
- 0 kudos
- 1118 Views
- 0 replies
- 1 kudos
Your Coworkers Don't Get Databricks? Here's How to Fix That.
We've all been there. You're excited about the lakehouse, you see the value clear as day, and then you try explaining it to a coworker and their eyes glaze over. Slides don't cut it. Documentation links get ignored. What actually works? Showing them...
- 1118 Views
- 0 replies
- 1 kudos
- 12952 Views
- 4 replies
- 6 kudos
The Medallion Architecture: Why Data Layers Matter for Modern Organisations
In today’s data-driven world, organisations are drowning in information. From customer transactions and IoT sensor readings to social media interactions and operational logs, the volume and variety of data continue to grow exponentially. Yet many org...
- 12952 Views
- 4 replies
- 6 kudos
Latest Reply
- 6 kudos
Thank you, @Louis_Frolio ! My next post is about Data Governance with Unity Catalog, stay tuned!!
- 6 kudos
- 7485 Views
- 5 replies
- 0 kudos
Unity cataog
Disaster recovery is possible in Unity catalog now?Means, for data level, we have enabled with geo redundancy, what about the objects, permissions, an other components in Unity catalog ? Can we restore the unity catalog metadata in another region ?
- 7485 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
Official product release in development will be available as PrPr in a few months.
- 0 kudos
- 1079 Views
- 0 replies
- 1 kudos
Managing Databricks CLI Versions in Your DAB Projects
If you are going with DABS into a production environment, a CLI version is considered best practice. Of course, you need to remember to bump it up from time to time. Learn more: - https://databrickster.medium.com/managing-databricks-cli-versions-i...
- 1079 Views
- 0 replies
- 1 kudos
- 1110 Views
- 0 replies
- 1 kudos
Distributed ML on Databricks Serverless
You can now run distributed ML (Spark MLlib in Python, Optuna tuning, MLflow Spark, Joblib Spark) on serverless notebooks/jobs and on standard clusters, not just dedicated ML clusters.It reuses the same Unity Catalog + Lakeguard stack you already use...
- 1110 Views
- 0 replies
- 1 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
