BI compatibility mode allows us to query Unity Catalog Metric Views from external BI tools. It allows Databricks to rewrite queries generated by tools like Power BI, ensuring complex metric view logic is executed correctly while appearing as standard...
Big shift coming for Azure Databricks usersStarting 30 September 2026, all new Azure Databricks workspaces will be Unity Catalog only.- No DBFS root.- No Hive Metastore.- No legacy runtimes below 13.3 LTS.- No “old way” of doing things.Disabling DBFS...
Hi guys,There was an email message sent by Azure Databricks with this information. So no direct link, just information sent by Databricks to users.
Hi everyone,I recently dealt with a frustrating scenario: a Databricks job that usually takes minutes ran for 18 hours without failing, quietly consuming compute and blocking downstream pipelines.The driver hadn't crashed, and the job hadn't failed—i...
Ever deleted a pipeline… and accidentally wiped out the data with it? Databricks just introduced a beta feature that lets you decouple pipelines from the tables they manage.Lakeflow Spark Declarative Pipelines were desinged with data-as-code approach...
Creating PowerPoint decks from data is usually manual and repetitive:Export charts → take screenshots → format slides → repeat.Not anymore.You can now generate a complete PowerPoint directly from a Databricks table — in one notebook run. What this do...
In Databricks Lakeflow Connect for MySQL (currently in public preview), Databricks recommends limiting each ingestion pipeline to around 250 tables, with validated testing up to 1 TB of snapshot data.However, in real-world enterprise scenarios, custo...
@antoalphi I think you have already answered in the first line itself - Public Preview -- meaning not full developed for general/real use. Hence it comes with limitations or bugs which will be covered in General Available. Though still you may consid...
Austin data community, this one looks worth attending. Databricks DevConnect Austin is happening on Tuesday, April 14, 2026, from 5:00 PM to 9:00 PM at Qubika Office in Austin. It is a technical meetup built for data and AI practitioners who want rea...
Great opportunity to connect with amazing and like minded minds.Would love to see more such events in India as well, specially in Lucknow, Noida, Bhopal -- Events Team
Core IdeaDon’t let “1 pipeline = 1 always-on cluster” become your cost trap.Instead, design for controlled parallelism + shared compute + smart grouping. A. Pipeline Sharding Strategy (Not Blind Splitting)Instead of randomly splitting 6,700 tables i...
MCP Servers on DatabricksGenerative AI is evolving rapidly, and one of the most exciting developments is standardizing how models interact with external systems. Let me walk you through how we got here and why the Model Context Protocol (MCP)—especia...
Insightful, detailed and incisive guide @VinayKumarB. Keep posting!
If you’ve ever worked with scheduled refreshes for Materialized Views or Streaming Tables, you probably know this pain. If your DDL-scheduled MV or ST refresh failed, nothing happened. No email, no alert, no indication that your data was stale (until...
Welcome back, @szymon_dybczak! That silent failure pain was real. These alerts are a massive win for everyone’s sanity. Thanks for the heads-up!
Databricks Clean Rooms are secure & governed collaboration environments that enable various organizations to run joint analytics without exchanging raw data eliminating sensitive data exposure. Its built-on Delta Sharing, Serverless and Unity Catalog...
I. IntroductionData pipelines are the lifeblood of modern data-driven organizations. However, even the most robust pipelines can experience unexpected issues: data corruption, erroneous updates, or sudden data drops. When these problems occur, quickl...
Great thought to use delta time travel to determine when data drift starts!But this only works as long as retention policies allow it. With vacuum and stricter runtime enforcement in newer dbx versions, older snapshots may not be there when you need ...
Hi everyone,I built Intelligence Studio, an open-source workbench that lets you browse, test, analyse, and integrate with 640+ Databricks REST APIs -- all from one interface. No more juggling docs, curl commands, Postman collections, and multiple bro...
Hey Team, in my last post I asked how much AI has actually changed your day to day, and the responses were fantastic. But let's talk about the other side for a minute. I'll go first — I've started second-guessing almost everything I see on social med...
AI tools generate code & pipelines that work functionally but ignore efficiency, scalability & cloud implications. I have bumped into belowCode generated by AI does a SQL cross join as its simple for most natural language queries that works but kills...
If you've ever worked on an insurance data warehouse, or really any warehouse where data arrives from different systems at different times, you know the pain of keeping things in sync. I spent years building data warehouses for a property and casual...
| User | Count |
|---|---|
| 85 | |
| 71 | |
| 47 | |
| 44 | |
| 42 |
