Hey Databricks Community! Join us for a free community-sponsored BrickTalk where we’ll explore Multi-Cloud Data Sharing with OpenSharing SecureConnect & Global Distribution. BrickTalks is a community event series where Databricks experts share real-w...
- 293 Views
- 0 replies
- 0 kudos
Wednesday
Databricks Advanced Learning Festival November 10 – December 8, 2026 Join us for a four-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whether you are ...
- 40039 Views
- 66 replies
- 12 kudos
10-08-2026
NOVEMBER · THE TOUR IS ON THE MOVE Nine stops. Three regions.One epic November. THIS MONTH'S ROUTE Munich → Atlanta → Amsterdam → London → Toronto → Los Angeles (Anaheim) → New York City → Milan → Tokyo A free, one-day event on how organizations...
- 3284 Views
- 1 replies
- 1 kudos
3 weeks ago
DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...
- 2506 Views
- 0 replies
- 2 kudos
4 weeks ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 4672 Views
- 0 replies
- 4 kudos
09-01-2026
Predicting patient disease risk and measuring quality of care are important to precision health. This Solution Accelerator demonstrates a machine learning workflow using healthcare data to identify clinical and demographic factors associated with dis...
- 154 Views
- 0 replies
- 0 kudos
Wednesday
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 1069 Views
- 0 replies
- 4 kudos
a month ago
Community Activity
- 28 Views
- 1 replies
- 0 kudos
Is Databricks Losing Its Charm in the Age of AI?
I've been working with Databricks for nearly eight years, and lately I've been reflecting on how our role as data engineers is evolving.One thing I've noticed is that development is increasingly becoming about prompting AI, troubleshooting issues, fi...
- 28 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@thekushalvishwa AI can write the pipeline in few seconds. It might not explain the DBU bill to finance. A timely post, and one I suspect many of us relate to.My take is that AI has made writing the code cheap, so our value has moved to the parts it ...
- 0 kudos
- 36 Views
- 0 replies
- 0 kudos
Free edition - serverless option stopped working
I am on free edition and suddently serverless option stopped working. Is there any restriction on free edition or something is down today?
- 36 Views
- 0 replies
- 0 kudos
- 56 Views
- 0 replies
- 0 kudos
Transactional apps on Databricks: Lakebase owns the decision, Delta gets the copy
A condensed version of the full article Beyond Analytics: Building Transactional AI Applications with Databricks and Lakebase. The working demo and its tests are in lakebase-patch-desk.The pipeline is built and the numbers are in Delta. Now business ...
- 56 Views
- 0 replies
- 0 kudos
- 291 Views
- 3 replies
- 0 kudos
Masking Columns for a Client Workspace When You Can’t Use Catalog Binding
Hi Everyone,I am Adnan and I am a Cloudera Platform expert recently joined the Databricks Community. I will be sharing all my implementations and experience I have gained as Big Data Expert and any solutions I implement to address my current business...
- 291 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
the fail-closed revoke is the part id take from this, a missing row-filter tag failing open is the one thats hardest to spot. is that check part of the sync job itself or a separate sweep?
- 0 kudos
- 56 Views
- 0 replies
- 0 kudos
Protecting Your Company Data When Integrating Coding Agents through Unity AI Gateway
Coding agents like Claude Code, Codex, OpenCode, or PI, started through the ug launcher, and integrated with foundation models through Unity AI Gateway can run with whatever Databricks permissions the developer already has through MCPs that is native...
- 56 Views
- 0 replies
- 0 kudos
- 63 Views
- 0 replies
- 0 kudos
Pushing a notebook with outputs to Github
I am experiencing an error when trying to commit output and Databricks notebook to GitHub. The error message states, "Error creating or updating commit outputs config file."Could you please assist me in resolving this issue?
- 63 Views
- 0 replies
- 0 kudos
- 147 Views
- 2 replies
- 0 kudos
Databricks best practices for ad-hoc analytics alongside CI/CD
Hi everyone,We currently use CI/CD (Databricks Asset Bundles + Git) for deploying notebooks, jobs and pipelines across DEV and PROD.However, we also have frequent ad-hoc requests from business teams (customer lists, segmentation, one-time analysis), ...
- 147 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Allowing read only DEV access to PROD catalogs is a fully supported capability in Unity Catalog but is not recommended. If you plan to go the read only route, the mechanism is direct and supported. Handling PII and sensitive customer data in this set...
- 0 kudos
- 200 Views
- 3 replies
- 1 kudos
How do you test data quality in a Delta Lake pipeline?
I'm exploring practical approaches to validating data quality in Databricks pipelines.For pipelines that ingest data into Delta tables, what checks do you typically automate at each stage?I'm particularly interested in handling:Schema changes and sch...
- 200 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
The right pattern is to treat validation as multi layered instead of a single gate. You can start with handling schema changes and schema evolution at ingestion. You can configure Auto Loader with a dedicated schema at the ingestion point to prevent ...
- 1 kudos
- 117 Views
- 1 replies
- 0 kudos
Your Thoughts on the Modern MI/BI/Engineer Role Description?
Hi all,First post for me….……….we are building the dream that is value enablement through Databricks and are around 6 months in to our journey. I am going to hire in someone to compliment and build upon the team who are already driving us through the ...
- 117 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Welcome, Mark. My view is that the hybrid exists, but I would be wary of writing it up as one job. The blend that seems most realistic to me is a BI developer who is strong in SQL and picks up a few engineering habits: version control, tests on the t...
- 0 kudos
- 940 Views
- 1 replies
- 2 kudos
What Is – Genie One vs a Genie Agent vs Genie Code? (And which one is for you)
On our last Genie community call (join the calls), one question came up that might be in the minds of a lot of people: "There's Genie One, Genie Agents and Genie Code. Which one is used for what? Which one should I be using?" It's a fair question. Al...
- 940 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
The front desk, specialist, builder split is an easy way to explain it to people. On your ownership question, my guess is it has to be shared, with the data team owning the agent as a thing (tables, instructions, who can edit it) and the business own...
- 2 kudos
- 590 Views
- 1 replies
- 0 kudos
Genie can now live in your favorite IDE!
You can now open a terminal in your project folder and type genie You now have a coding agent that can read your local files, run commands, and query your Unity Catalog tables in one conversation. That is Genie Code CLI, now available in Beta. Most...
- 590 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Thanks for the walkthrough. One line on the docs page is easy to skim past: model access goes through Unity Gateway and your Databricks permissions limit what it can reach in the workspace, but file edits and commands on your own machine are not gove...
- 0 kudos
- 431 Views
- 1 replies
- 1 kudos
What Is – an ontology? (How Genie learns your business like a new joiner)
At Big Data London this year, I spent two days on the Unity Catalog booth and the main topic of the booth was 'Databricks Ontology. So "what is an ontology?" - here's the plain-English answer. The short answer: Think of a new joiner On day one, a new...
- 431 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
The new joiner picture works well. One thing from the ontology docs that the post doesn't mention: the snippets Genie learns can't be edited directly. If it picks up a wrong definition, you either fix the query or dashboard it came from, or write the...
- 1 kudos
- 353 Views
- 2 replies
- 0 kudos
Follow-Up on Exam Suspension Appeal – Request #01052422 | No Update Received
Hi @Cert-Team and @cert-ops,I submitted my appeal regarding my Databricks certification exam suspension on Thursday, October 22, 2026. My exam was suspended immediately when I clicked on “Start Exam,” before I could access or answer any questions.The...
- 353 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello Databricks Support Team,I am following up again regarding my suspended Databricks Associate Data Engineer certification exam appeal.I have already raised multiple support requests and sent several follow-up emails, but unfortunately, I have not...
- 0 kudos
- 219 Views
- 3 replies
- 4 kudos
Resolved! SparkSession.getActiveSession() is Suddenly Broken in Python Multi-threading
Hello everyone,Around 3PM EST yesterday (2026-11-12), dozens of Lakeflow jobs started failing on both classic and serverless compute with distinct symptoms1. On classic compute (DBR 15.4), reading a file from S3 would result in a PATH NOT FOUND error...
- 219 Views
- 3 replies
- 4 kudos
Latest Reply
- 4 kudos
Hello @ifishbein, I dug in and here is what I found. The serverless behavior is explainable, and your workaround is the right one. Serverless runs on Spark Connect, and in the Spark Connect flavor of PySpark SparkSession.getActiveSession() is backed ...
- 4 kudos
- 218 Views
- 4 replies
- 0 kudos
Lakebase Data API
We need to bring the json data from external API and need to load it into lakebase API tables.Please suggest in connecting this.
- 218 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Jack17,One small addition to the service principal advice above, because it makes testing a lot easier. Since 12 November, OAuth on-behalf-of for service principals is generally available, so you can test the Data API locally as the same service ...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1517 | |
| 1255 | |
| 1190 | |
| 961 |
Latest from our Blog
From Kyriba to Lakehouse: A Near-Real-Time Treasury Data Pipeline on Databricks
The two-speed problem of treasury data Corporate treasury asks the same question at two speeds. Where is our money right now? And where was it at the close of business? For many treasury teams the ans...
How to Query Unity Catalog Views from External Engines
Summary External engines can now query Unity Catalog views. Views over Delta tables are served through the Unity REST API (Generally Available). Views over Iceberg tables are served through the Iceber...
Private Connectivity from Databricks Serverless on AWS to Your On-Prem Data Center
How to connect Databricks serverless workloads to on-premises databases and services over a private network path, using Network Connectivity Configurations and AWS PrivateLink. Why private connectivit...
Reinforcement Learning on Databricks: Building a Contextual Bandit
Is MLflow really built only for supervised learning, or can it handle something like reinforcement learning? A customer asked me exactly this. They're running a contextual-bandit-style system on-premi...
