Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, October 29! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 559 Views
- 0 replies
- 1 kudos
2 weeks ago
Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, November 5! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 129 Views
- 0 replies
- 1 kudos
yesterday
DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...
- 359 Views
- 0 replies
- 0 kudos
Wednesday
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 2862 Views
- 0 replies
- 3 kudos
09-01-2026
Community Champion • September 2026 Meet Venugopal Dabbara Every month, our Community Champion Program recognizes a member who makes the Databricks Community better for everyone, through their knowledge, their willingness to help, and their commitmen...
- 788 Views
- 3 replies
- 6 kudos
2 weeks ago
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
- 93 Views
- 1 replies
- 1 kudos
yesterday
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 426 Views
- 0 replies
- 4 kudos
a week ago
Community Activity
- 68 Views
- 3 replies
- 0 kudos
Materialized View with Incremental Refresh - Change Data Feed required on source tables?
Question:To create a materialized view with incremental refresh, is the Delta Lake feature Change Data Feed required on all of the source tables that the materialized view consumes? Or, is it not required for the creation of the materialized view, bu...
- 68 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @johnb1, Following up with the doc references behind the two answers above, and thank you for the test runs in the second one. Change data feed is a performance recommendation. No documented check, error code or refresh rule makes it a conditio...
- 0 kudos
- 106 Views
- 3 replies
- 0 kudos
Why Is Outbound Internet Access Restricted on GCP Databricks Serverless?
I'm using Databricks on GCP, apparently on the Premium tier, and Serverless compute blocks requests to public websites such as Google and data.gov.au.I would expect a paid Databricks workspace to support basic outbound HTTPS requests.Unfortunately, t...
- 106 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
I've spent several days troubleshooting this issue, trying various workarounds, including VPNs, proxies, and SSH tunneling, just to keep my project moving.The logs confirm that Databricks Serverless is actively blocking outbound DNS requests (DROP, r...
- 0 kudos
- 831 Views
- 6 replies
- 6 kudos
Resolved! Unity Catalog v.2.0?
Are there plans for a v.2.0 of Unity Catalog? I find the organization of tables in databricks to be very constrained and inflexible. Given that UC tables have their origins in data lakes, you would think they would have brought a lot more flexibile...
- 831 Views
- 6 replies
- 6 kudos
Latest Reply
- 6 kudos
Fair points, especially for data coming from systems with richer naming. I can't speak to the roadmap, but here's what works for us today:Aliases: Create views in a consumer-facing catalog, like reporting.sales.outside_sales, that point to the real t...
- 6 kudos
- 13 Views
- 0 replies
- 0 kudos
- 13 Views
- 0 replies
- 0 kudos
- 2288 Views
- 2 replies
- 2 kudos
How to Unit Test Spark Declarative Pipelines
Every engineer who ships application code writes unit tests for it. Yet the data pipeline that feeds the same application is often left with a manual run and a dashboard that someone eventually notices is wrong. The gap has been more about tooling th...
- 2288 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Great article. The native TestPipeline and test_spark APIs are a promising step towards first-class unit testing for SDP.One potential enhancement would be execution outside the Lakeflow Pipelines Editor.As documented, native test execution is curren...
- 2 kudos
- 27 Views
- 0 replies
- 0 kudos
Declarative Automation Bundles
Databricks Declarative Automation Bundles Abri um pull request que apagava um job. O plan mostrou “delete jobs.sample_job”, o check ficou verde e o merge liberou.Se eu clicasse, o job sumia de HML na hora e, depois da aprovação, de PRD também.O CLI ...
- 27 Views
- 0 replies
- 0 kudos
- 69 Views
- 1 replies
- 0 kudos
Do We Need a New Role in Data Organizations — The Semantic Steward?
As data platforms evolve from managing data to managing business meaning, is it time to introduce a new role: Semantic Steward?Traditionally, data governance has focused on data quality, ownership, security, lineage, and compliance.But with the rise ...
- 69 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Venkat, your point about different business teams defining 'Revenue' or 'Active Customer' differently is a classic source of friction. I've seen this in almost every implementation, and you're right that even with perfect data quality, inconsistent s...
- 0 kudos
- 87 Views
- 5 replies
- 0 kudos
Salesforce - "Unexpected error occurred" while connecting with Databricks
Hi everyone,I am using Salesforce trail version. I am getting error which shows "Unexpected error occurred" while connecting it with Databricks free edition, However, I already passed the necessary OAuth with the credentials.Please advise what action...
- 87 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
@J-DataCould you as-well post the Error details to see if its connection issue?
- 0 kudos
- 142 Views
- 5 replies
- 1 kudos
Feature request: let DABs alert resources load query_text from an external .sql file
Hi everyone,We manage SQL alerts with Declarative Automation Bundles and like having them in version control. One thing we'd like to improve: the alert's SQL query has to be written inline in the YAML resource definition (query_text).The problemOur a...
- 142 Views
- 5 replies
- 1 kudos
Latest Reply
- 1 kudos
Short answer: You can get "SQL in a .sql file" today, without waiting for a query_file field, by defining the alerts with Python for Declarative Automation Bundles [1]. A small load_resources() function reads each .sql file and passes its contents as...
- 1 kudos
- 78 Views
- 0 replies
- 0 kudos
CUSTOMER STORY | bsport builds an agent-ready data stack with Spark Declarative Pipelines
“Not having a standard framework was terrible. Now we have a foundation that helps us scale — with governance, lineage and data all in one place. As we bring AI into more of our workflows, that foundation becomes even more important. That’s always be...
- 78 Views
- 0 replies
- 0 kudos
- 68 Views
- 1 replies
- 0 kudos
Serverless compute charge on managed ingestion pipeline despite 100 DBUs/day not reached
Implementing the managed ingestion pipeline for D365 using Lakeflow Connect. No technical issues but confusion when it comes to the charges attributed since Lakeflow Connect on our premium workspace is eligible for 100 DBUs per day. When reviewing sy...
- 68 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, I've just done some digging into this and as far as I can tell there shoulnd't be any charges for the Lakeflow Ingestion until you hit the 100dbu limit. The only thing I want to clarify is that the 100DBU limit is a total for all ingestion pipeli...
- 0 kudos
- 93 Views
- 1 replies
- 1 kudos
Solution Accelerator Series | Responsible Gaming
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
- 93 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
excellent
- 1 kudos
- 11096 Views
- 3 replies
- 3 kudos
Why database/table names are in lower case ?
Hello,When I run this code :CREATE DATABASE BackOfficeI see the database like this :backofficeWhy everything is in lower case ?Is it possible to configure Databricks in order to keep the real name ?Thanks.
- 11096 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
It is managed by hive metastore as you can put it in different databases is saver this way as some database are Case Sensitive and some not (you can easily test it with standard WHERE syntax).Probably you could change it with some hive settings but i...
- 3 kudos
- 77 Views
- 0 replies
- 0 kudos
AI/BI + Genie Launchpad (Nov 3-5) | Free Hands-On Training
Free · Hands-on · Instructor-led AI/BI + Genie Launchpad From dashboards your team can trust to answers in plain language with Genie. A three-day arc of live, hands-on sessions, each built for a different role. Join only the day or days that fit you...
- 77 Views
- 0 replies
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1504 | |
| 1255 | |
| 1188 | |
| 961 |
Latest from our Blog
AI Functions for Synthetic Test Data: Generate Realistic Datasets in Pure SQL
The views expressed in this post are my own and do not represent those of my employer. Not long ago, generating realistic synthetic data meant building custom pipelines with GANs, Transformers, or RN...
Databricks maintenance and cleanup: Visualize, Clean, and Log with Declarative Automation Bundles
Introduction Managing Databricks environments can quickly become overwhelming as jobs, AI Search indexes, and pipeline artifacts accumulate, making manual cleanup, logging, and compliance difficult. T...
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 2)
Part 1 introduced the pattern for verifiable visual AI. Let the model find the evidence, use deterministic code to measure it, apply governed rules to produce a pass, review, or fail verdict, and send...
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 1)
AI adoption has moved much faster than measurable business results. McKinsey reports that 88% of organizations now use AI in at least one business function, yet only 39% see a bottom-line effect. A co...
