Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, November 5! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 636 Views
- 1 replies
- 2 kudos
2 weeks ago
NOVEMBER · THE TOUR IS ON THE MOVE Nine stops. Three regions.One epic November. THIS MONTH'S ROUTE Munich → Atlanta → Amsterdam → London → Toronto → Los Angeles (Anaheim) → New York City → Milan → Tokyo A free, one-day event on how organizations...
- 1448 Views
- 1 replies
- 0 kudos
a week ago
DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...
- 1132 Views
- 0 replies
- 2 kudos
2 weeks ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 3772 Views
- 0 replies
- 4 kudos
09-01-2026
Free · Hands-on · Instructor-led AI/BI + Genie Launchpad From dashboards your team can trust to answers in plain language with Genie. A three-day arc of live, hands-on sessions, each built for a different role. Join only the day or days that fit you...
- 822 Views
- 0 replies
- 1 kudos
2 weeks ago
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
- 572 Views
- 1 replies
- 5 kudos
2 weeks ago
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 766 Views
- 0 replies
- 4 kudos
3 weeks ago
Community Activity
- 65 Views
- 1 replies
- 0 kudos
Update Ingestion pipleline to use pager duty with API
Hi,I have been working to update a cdc ingestion pipeline to add a notification to it on failure. We have had two instances where a maintenance task by vendor as tripped up CDC with some grief.I reviewed all the information and I am getting an error ...
- 65 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi! I think the errors come from how the update is sent, not from the notification block itself.1. PUT /api/2.0/pipelines/{id} replaces the whole spec, it doesn’t merge. That’s why it asks for gateway_definition. The 'schema' field is illegal error u...
- 0 kudos
- 33 Views
- 0 replies
- 0 kudos
403: Permission Denied for certain foundational models
Hello,I have an individual account (Azure Databricks, Premier, East US)without any support contracts, and I am using it solely for personal projects. I am getting this error whenever I am trying to use the GPT and/or Claude models (system.ai.claude-s...
- 33 Views
- 0 replies
- 0 kudos
- 25 Views
- 0 replies
- 0 kudos
Gating DEEP CLONE backups: what saved time, what didn’t, and the bug that almost slipped by
I’ve spent a good part of this year on Unity Catalog backups at scale: thousands of Delta tables, nightly DEEP CLONE into a separate backup catalog, and a backup window that kept getting longer. The obvious fix was “only clone tables that actually ch...
- 25 Views
- 0 replies
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
One PDAL init script for every Databricks Runtime (and no apt)
I needed PDAL on Databricks for a LiDAR processing workload. PDAL is a C++ library with a long native dependency chain (GDAL, PROJ, HDF5, a recent libstdc++), and the Python bindings have to match the interpreter exactly. The usual approach, an apt-g...
- 22 Views
- 0 replies
- 0 kudos
- 30 Views
- 0 replies
- 0 kudos
Backing up Unity Catalog volumes without melting the driver
Tables are the easy part of a Unity Catalog backup, because you have DEEP CLONE. Volumes have no equivalent. When I started on a self-service backup tool this year, the first version backed volumes up the way I suspect most people do: list the files ...
- 30 Views
- 0 replies
- 0 kudos
- 26 Views
- 0 replies
- 0 kudos
Lakeflow Expectations: Dropping Bad Rows Can Hide Real Sales
A practical data quality lesson: choose the response to a failed rule based on its business impact.Consider a hypothetical retailer using Azure Databricks. An incoming order has a valid order ID and sales amount, but its customer ID is missing.A rul...
- 26 Views
- 0 replies
- 0 kudos
- 20 Views
- 0 replies
- 0 kudos
Building on Lakebase | Episode 3: Branch, Test, and Ship: Release Changes Safely using Branching
Welcome to Building on Lakebase, a four-part series on building transactional apps and AI agents on Lakebase, a fully managed serverless PostgreSQL database built by Databricks. Over the four episodes led by Michelle Deng and Benjamin Nwokeleme, Solu...
- 20 Views
- 0 replies
- 0 kudos
- 30 Views
- 0 replies
- 0 kudos
Models Should Be Config, Not Copies: A Config-Driven MLOps Blueprint on Databricks
Banking · Insurance · Credit Risk · Any Team Running More Than One Model — From Copy-Pasted Pipelines to One Shared Pipeline and One YAML per ModelBy @SrinuB in collaboration with @sridharplv Your first model went to production beautifully. Your fif...
- 30 Views
- 0 replies
- 0 kudos
- 181 Views
- 3 replies
- 0 kudos
Real time Streaming to AI Engine at Databricks
Hi there,Usercase: Fraud Detection Technology : Databricks, Kafka, Microsoft Foundry, Dashboard showing real-time reportsAsk: Please let me know if any reference or technology details to achieve 1-2 milli second (Max) latency using above technology s...
- 181 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @abhietc31, the fraud detection Solution Accelerator scores Kafka transactions in real-time mode and feeds a live Databricks Apps dashboard from Lakebase. For your own pipeline, the Lakebase sink writes real-time mode output to a Lakebase table wi...
- 0 kudos
- 456 Views
- 3 replies
- 0 kudos
Unable to Serve Gemini 3.8 Flash on Azure Databricks – East US
I am working with an Azure Databricks workspace in the East US region and am trying to use the Gemini 3.8 Flash model through Databricks Model Serving.The model is visible in the workspace under the system AI models as:system.ai.databricks-gemini-3-8...
- 456 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @krSindhu, on Azure Databricks, Gemini 3.8 Flash needs both ADI Services, which require eligibility and enablement, and cross-geography routing. Check that the Integration with ADI Services toggle is enabled. Confirm that Enforce data processing...
- 0 kudos
- 33 Views
- 0 replies
- 0 kudos
Exam Suspended before start (Kryterion) - Ticket #01054624
Hi everyone,I'm reaching out here hoping a community moderator or Databricks staff member can help escalate my support ticket.On October 25, 2026, I was scheduled to take the Databricks Certified Data Engineer Associate exam. I paid the 246 USD certi...
- 33 Views
- 0 replies
- 0 kudos
- 65 Views
- 2 replies
- 0 kudos
Best approach for moving Alteryx workflows to Databricks ?
Hey folks, We're planning to migrate our ETL pipelines from Alteryx to Databricks.Many of our workflows use iterative macros and complex joins, so manually rewriting everything into PySpark/Spark SQL will take a while.For those who have made this tra...
- 65 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Great answer from @Harshad_B .Several things to addd: there is No native one-click converter exists for Alteryx-to-Databricks migration, so the you need to follow a structured approach covering tooling, iterative macro translation, target architectur...
- 0 kudos
- 33 Views
- 0 replies
- 0 kudos
How to size your Genie bill (Genie Agent, Genie Code) before the free period ends (31 January 2027)
31 January 2027 is closer than it looks. Until then, every question your people ask Genie One or a Genie Agent is free. From 1 February, it's billed like the rest of Genie: a free monthly allowance per person, then pay-as-you-go. If nobody has worked...
- 33 Views
- 0 replies
- 0 kudos
- 32 Views
- 0 replies
- 0 kudos
Structured Streaming: Choose Watermarks Around Real Arrival Delays
Structured Streaming: Choose Watermarks Around Real Arrival DelaysA practical streaming lesson: measure how late business events arrive before choosing a watermark threshold.Consider a hypothetical retailer using Azure Databricks to calculate sales i...
- 32 Views
- 0 replies
- 0 kudos
- 94 Views
- 0 replies
- 0 kudos
Governed Writes with SQL SECURITY DEFINER for Stored Procedures
SQL Stored Procedures in Databricks now support SQL SECURITY DEFINER, and it is Generally Available on SQL warehouses, Serverless compute and All-Purpose clusters (DBR 19+). A SECURITY DEFINER procedure runs with its owner's privileges, not the calle...
- 94 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1506 | |
| 1255 | |
| 1189 | |
| 961 |
Latest from our Blog
Governed Writes with SQL SECURITY DEFINER for Stored Procedures
SQL Stored Procedures in Databricks now support SQL SECURITY DEFINER, and it is Generally Available on SQL warehouses, Serverless compute and All-Purpose clusters (DBR 19+). A SECURITY DEFINER procedu...
Assume, don't inherit. Putting Databricks RBAC to work.
Access problems in Databricks are rarely about missing permissions. More often they are about people carrying every permission they have ever been given into every session. Someone gets added to a fin...
Geospatial Reasoning with Genie Agents: Competitor Impact in Your Trade Area (Part 1)
As we start building agents that deliver business value, it is inevitable that subsets of the data an agent needs are geospatial in nature. When agents are left to make geospatial decisions on their o...
Serving the Lakebase Data API Through Azure API Management
Why use APIM with Data API? The Data API is one of the underrated features of Lakebase. Traditionally, teams that wanted to share a database over HTTP built a layer of REST services in front of it. Th...
