Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, October 29! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 228 Views
- 0 replies
- 0 kudos
Monday
DATABRICKS Asia-Pacific Learning Festival Free, virtual, hands-on Databricks training 21 & 22 October 2026 Virtual, labs included Free Ready to level up your Databricks skills? The Databricks Asia-Pacific Learning Festival is back, and ...
- 1654 Views
- 1 replies
- 1 kudos
a week ago
FOR DATABRICKS LEARNERS Two free ways to grow yourDatabricks skills Both are free and live in the Databricks Community. Bookmark the one that fits where you are right now - or use both. 01 LEARNING PATHS Your Path to Certification What it is:...
- 1800 Views
- 3 replies
- 6 kudos
3 weeks ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 2439 Views
- 0 replies
- 3 kudos
09-01-2026
Free · Hands-on · Instructor-led AI/BI + Genie Launchpad From dashboards your team can trust to answers in plain language with Genie. A three-day arc of live, hands-on sessions, each built for a different role. Join only the day or days that fit you...
- 695 Views
- 0 replies
- 0 kudos
Friday
Community Champion • September 2026 Meet Venugopal Dabbara Every month, our Community Champion Program recognizes a member who makes the Databricks Community better for everyone, through their knowledge, their willingness to help, and their commitmen...
- 579 Views
- 3 replies
- 6 kudos
a week ago
Genome-wide association studies help identify genetic variations associated with a disease or trait. The Genome-Wide Association Studies Solution Accelerator builds on Glow, an open source project for genomic analysis, to provide a fast, scalable and...
- 112 Views
- 0 replies
- 1 kudos
Tuesday
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 119 Views
- 0 replies
- 3 kudos
yesterday
Community Activity
- 15 Views
- 0 replies
- 0 kudos
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 2)
Part 1 introduced the pattern for verifiable visual AI. Let the model find the evidence, use deterministic code to measure it, apply governed rules to produce a pass, review, or fail verdict, and send anything uncertain to a human. In a demo, that ca...
- 15 Views
- 0 replies
- 0 kudos
- 76 Views
- 4 replies
- 0 kudos
Datavault implemention on Databricks
Hi everyone,I'm building a Data Vault 2.0 architecture entirely natively in Databricks from scratch. We have specifically decided not to use dbt.While Raw Vault ingestion is straightforward with Spark/Delta, I'm looking for architectural advice on co...
- 76 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Great points from @balajij8 on the SDP/LDP approach. I've worked on a few Data Vault implementations, and these are the things that tend to bite when the Business Vault grows:1. Hash keys: standardize before hashing. Trim, upper-case and use a fixed ...
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 1)
AI adoption has moved much faster than measurable business results. McKinsey reports that 88% of organizations now use AI in at least one business function, yet only 39% see a bottom-line effect. A common bottleneck in closing that gap is trust. You ...
- 22 Views
- 0 replies
- 0 kudos
- 141 Views
- 2 replies
- 0 kudos
Resolved! Service principals created by Genie App builder are not visible in Databricks UI
I created a Chat Assistant application using Databricks Genie App Builder, which automatically created a dedicated service principal for the application.The service principal requires access to the relevant Genie Space and catalog to run successfully...
- 141 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Bhawana,I'm running into a similar issue except the solution provided does not work - the service principal created by the Genie App Builder doesn't exist in the Account Console. I've tried looking it up by both name and application ID, but I can'...
- 0 kudos
- 11 Views
- 0 replies
- 0 kudos
External location Manage permission
Hi team, I'm trying to update the owner of an external location and I see the following message:Requires permission MANAGE on external location 'location_name'The owner of the resource, the user that created it doesn't have that permission? How is th...
- 11 Views
- 0 replies
- 0 kudos
- 28 Views
- 1 replies
- 0 kudos
Zerobus Variant Data not Processed as Expected (json data destined for Variant column).
Lets say I have a table receiving zerobus data (json) over HTTP/rest.That last VARIANT is "ValueJson".If I want to submit data to it, I would expect to be able to use normal json formatting:[ { "a": "E", "b": 123, "c": "xy...
- 28 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
I found docs that say I'm supposed to rely on json-encoded-json: https://docs.databricks.com/aws/en/ingestion/zerobus-concepts#supported-data-typesThat is pretty unfortunate. I guess it isn't the end of the world, but it gives a dirty feeling when p...
- 0 kudos
- 27 Views
- 0 replies
- 0 kudos
Disable auto-scroll on Databricks Genie Code
Hello, I frequently use Genie Code when developing, particularly to ask Genie questions about certain lines or sections of code. The responses are great, but because they are often long, the chat auto-scrolls while text is being generated. Is there a...
- 27 Views
- 0 replies
- 0 kudos
- 24 Views
- 0 replies
- 0 kudos
Zerobus rescue (VARIANT) not working. No Preview Available.
Has anyone tried using the rescue feature for json in zerobus-ingest? It seems like a very significant feature, assuming the schema incoming json is even 1% outside the control of team that may be ingesting on the databricks side. There needs to be ...
- 24 Views
- 0 replies
- 0 kudos
- 18 Views
- 0 replies
- 0 kudos
Pattern: Sharing Databricks output with an AKS app through an external volume and managed identity
Scenario A transaction app runs on AKS. It is the only component with a connection to MongoDB Atlas. Databricks processes the data, and the app has to pick up the results and write them back to Atlas. The requirements are no PATs, no CLI, no Databric...
- 18 Views
- 0 replies
- 0 kudos
- 32 Views
- 0 replies
- 2 kudos
What Is – a system table? (Your Databricks bank statement)
Clients ask me the same three questions all the time: What are system tables? What are they about? And what can I actually do with them? I recently came across a Databricks blog that answers the last one brilliantly: Top 5 System Table Queries for U...
- 32 Views
- 0 replies
- 2 kudos
- 243 Views
- 1 replies
- 4 kudos
Built a live Helsinki tram tracker on Databricks with Lakeflow, Lakebase, Genie and Apps
Hi everyone! I built a near-real-time app that follows every tram and metro train in Helsinki using only Databricks: no Kafka, no extra servers, and the whole thing deploys from one repository.What the app doesA live map of every tram and metro train...
- 243 Views
- 1 replies
- 4 kudos
Latest Reply
- 4 kudos
Thanks for sharing this insightful app
- 4 kudos
- 86 Views
- 1 replies
- 2 kudos
Should I wait for the tram? Adding agents to a live Helsinki transit app on Databricks (part 2)
Hi everyone,Part 2 of my Helsinki live transit project is out. In part 1 I built a Databricks App that follows every tram and metro train in Helsinki in real time: a Lakeflow pipeline on HSL's live MQTT feed, Lakebase behind the live map, and a Genie...
- 86 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Thanks for sharing this insightful app
- 2 kudos
- 33 Views
- 1 replies
- 0 kudos
auto.arima in R keeps running without end
I have a time-series that I am trying to model using auto.arima in R. Upon execution on a cluster runtime with 96GB memory and 48 cores, I observed that the code kept running without any end in sight. It ran for over a day without finishing.Curious, ...
- 33 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Okay, this is resolved. Windows doesn't allow forking, so parallel = TRUE was having no effect. But Databricks runtime runs on Linux, which enables forking. So keeping parallel = TRUE seems to be having the opposite effect than the anticipated one: d...
- 0 kudos
- 69 Views
- 1 replies
- 0 kudos
Possible bug in Lakeflow Connect SQL Server ingestion with GUID primary keys – Missing records
Hi everyone,We've encountered a strange issue with Databricks Lakeflow Connect query-based ingestion from SQL Server, and I'm wondering if anyone else has experienced something similar.We have a SQL Server table with 1,448 records and a uniqueidentif...
- 69 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@volodymyr_b The SQL Server connector reference confirms that UNIQUEIDENTIFIER is automatically converted to STRING, which makes the ordering behaviour you've identified worth investigating.I ran a few independent test using SQL Server and Databricks...
- 0 kudos
- 60 Views
- 1 replies
- 0 kudos
Discount vouchers
I have completed Databricks certified Data engineer associate exam last month at full price. Are there any discount vouchers for the Databricks certified Data engineer professional exam or the Databricks certfified Machine language associate exams.
- 60 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @tanmay_ai, Databricks offers 50% certification vouchers during its Learning Festival events.The recent Virtual Learning Festival has concluded, so please plan to participate in the upcoming November event. All the details, eligibility requirem...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1498 | |
| 1254 | |
| 1188 | |
| 961 |
Latest from our Blog
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 2)
Part 1 introduced the pattern for verifiable visual AI. Let the model find the evidence, use deterministic code to measure it, apply governed rules to produce a pass, review, or fail verdict, and send...
Locate, Measure, Judge: A Pattern for Verifiable Visual AI on Databricks (Part 1)
AI adoption has moved much faster than measurable business results. McKinsey reports that 88% of organizations now use AI in at least one business function, yet only 39% see a bottom-line effect. A co...
Minimize disruptions for continuous pipelines with Maintenance Windows
Minimize disruptions for continuous pipelines with Maintenance Windows In an agentic world, it is more important than ever to make decisions using the latest available data. To achieve this, data engi...
Introducing session restore for serverless jobs
Serverless jobs make it easy to run production notebook workloads at scale, but until now, the state of a run was gone once it ended. If a pipeline ran for hours and failed on the last step, the varia...
