REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 295 Views
- 0 replies
- 0 kudos
Tuesday
Databricks Advanced Learning Festival September 16 – October 14, 2026 Join us for a four-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whether you are...
- 14147 Views
- 19 replies
- 17 kudos
2 weeks ago
Virtual Event Time: AMER Sept 23 / 9 AM PTEMEA Sept 24 / 9 AM BST / 10 AM CESTAPJ Sept 24 / 9:30 AM IST / 12 PM SGT / 1 PM JST/KST / 2 PM AEST (Japanese and Korean captions available) Register for the Workshop Note: Marking RSVP on this Community ...
- 191 Views
- 0 replies
- 0 kudos
Wednesday
Databricks EMEA Learning Festival Hands On Virtual Training 29–30 September 2026 · 09:00 BST / 10:00 CEST · Fully virtual · Free to join Ready to accelerate your journey on Data & AI? Join us for the EMEA Learning Festival. A two-day, hands-on vir...
- 1986 Views
- 0 replies
- 2 kudos
4 weeks ago
Our Community Champion Program has always been about recognizing individuals who consistently show up to support others, share their expertise, and strengthen the Databricks Community. Alongside our customers, we’re equally proud to recognize the inc...
- 2830 Views
- 5 replies
- 6 kudos
4 weeks ago
Subscriber attrition can be difficult to anticipate. The Subscriber Churn Prediction Solution Accelerator shows how to analyze behavioral data, identify subscribers at increased risk of cancellation, and use machine learning to estimate churn likelih...
- 157 Views
- 0 replies
- 2 kudos
Tuesday
The Weekly Digest • August 24 – 30 Community Pulse A new week, a fresh wave of articles, answers, and discussions. Here's your weekly catch-up. This Week's Community Stars The members who showed up and made a difference this week : @balajij8...
- 154 Views
- 0 replies
- 1 kudos
Thursday
Community Activity
- 4 Views
- 0 replies
- 0 kudos
Recommendation for Data Reconciliation Frameworks (Legacy vs. Migrated Validation)
Hi Team,We need to perform a data quality check between an legacy table and a newly migrated table generated by refactored code. Specifically, we want to validate:Schema & Volumetrics: Total row counts, column counts, and column data type distributio...
- 4 Views
- 0 replies
- 0 kudos
- 3926 Views
- 8 replies
- 6 kudos
Resolved! Unexpected Script Execution Differences on databricks.com vs Mobile-Triggered Runtimes
I’m noticing some unusual inconsistencies in how scripts execute on databricks.com compared to when the same workflow is triggered through a mobile-based API. On Databricks, the script runs perfectly when executed directly inside a cluster notebook. ...
- 3926 Views
- 8 replies
- 6 kudos
Latest Reply
- 6 kudos
This kind of difference can be surprisingly easy to miss because the same notebook may behave differently depending on how it is launched. In Databricks, compute type, runtime, permissions, parameters, and whether the execution is interactive or a jo...
- 6 kudos
- 2049 Views
- 7 replies
- 7 kudos
FAQ for Advanced Learning Festival September 16 - October 14 2026
Frequently Asked Questions Databricks Advanced Learning Festival | September 16 - October 14 2026 General Where can I ask questions while going through the self-paced material? The Databricks Academy Learners Group page is a forum within the Data...
- 2049 Views
- 7 replies
- 7 kudos
Latest Reply
- 7 kudos
What If we've have attempted the certification exam & couldn't clear it in the 1st attempt. Are we once again eligible to get one more 50% discounted voucher in-order to retake the certifications once again? Although do we need to once again go throu...
- 7 kudos
- 76 Views
- 3 replies
- 3 kudos
Genie - Cost monitoring and usage
Hi Team,Genie is widely used within our department, and we would like to establish appropriate governance around its usage. Do we have any existing guidelines or recommended practices for this? In particular, we are looking for guidance on:Cost monit...
- 76 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
Hey @Sanjeeb2024, Good to note Genie usage in your org. Databricks does have solid governance tooling for Genie, and it's matured quite a bit since billing went live on July 8, 2026. Here is what you need to know. The source of truth for Genie spend ...
- 3 kudos
- 98 Views
- 2 replies
- 0 kudos
Does enabling Photon improve performance while lowering compute costs?
Please refer screen shot and help in clarifying if enabling Photon lead to lower execution costs?
- 98 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @neerajdubey_86, Photon will almost certainly make your queries faster, but whether it also makes them cheaper depends on your workload. Photon is Databricks native vectorized query engine. It replaces the JVM-based Spark execution layer with ...
- 0 kudos
- 40 Views
- 0 replies
- 0 kudos
I Stopped Sending Every Data Engineering Task to an LLM - A Cost Aware Routing Pattern on Databricks
AI is becoming part of almost every data engineering workflow.We are asking AI to generate SQL, PySpark pipelines, data-quality rules, unit tests, technical specifications and even complete project structures. The results can be impressive, but there...
- 40 Views
- 0 replies
- 0 kudos
- 42399 Views
- 37 replies
- 34 kudos
Resolved! Free Certification voucher
looking for free certification voucher
- 42399 Views
- 37 replies
- 34 kudos
Latest Reply
- 34 kudos
You can check for the upcoming Learning Festivals for discount vouchers in June-August or October-November. You can check the upcoming learning events here
- 34 kudos
- 41 Views
- 0 replies
- 0 kudos
Databricks-Native AI Agent for Job Incident Detection, RCA & Safe Remediation
I’m exploring an architecture for a Databricks-native AI Agent for intelligent Spark job incident detection, root cause analysis (RCA), and safe remediation, and I would love your technical feedback.The ProblemA common operational challenge is the "S...
- 41 Views
- 0 replies
- 0 kudos
- 50 Views
- 0 replies
- 0 kudos
Building Custom Agents on Databricks:LangGraph, Atlan-Grounded Routing, and Per-Run Cost with MLflow
The problem: rethinking routing in multi-agent systems on DatabricksDatabricks native supervisor agent pattern is a solid default for orchestrating multiple Genie spaces and tools but it comes with a routing constraint worth examining. The superviso...
- 50 Views
- 0 replies
- 0 kudos
- 41 Views
- 0 replies
- 0 kudos
Evaluating Unity AI Gateway for Enterprise Cost Control
A rising AI bill can have several explanations. More work is being completed. Documents are getting longer. Failed requests are being repeated. Or a model chosen during a proof of concept has become the default for every task.Each explanation calls f...
- 41 Views
- 0 replies
- 0 kudos
- 76 Views
- 1 replies
- 1 kudos
Databricks vs Snowflake Pyspark Performance
Hi experts, now the competition between cloud providers are fierce and brutal , I come across this post which compares Pyspark performance on databricks and Snowflakehttps://l1nk.dev/1y33h0z although the metics tested are not rigorous and exhaustive...
- 76 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@Sam500 You can suggest PySpark on Databricks as Databricks is the original home of Apache Spark - the creators of Spark built Databricks optimizing it further and released it as a high performant platform for use. The Photon engine provides vectoriz...
- 1 kudos
- 72 Views
- 2 replies
- 0 kudos
Databricks Apps - Deployment process - Need help !!
Hi all,Do we have any guidelines or a CI/CD approach for deploying a Databricks app from one environment (e.g., DEV) to PROD?We are currently facing the following challenge:We created a Databricks app in the DEV environment through the UI. At runtime...
- 72 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
@Sanjeeb2024 Definitely DABs are good approach here. One thing I would add here is to keep the deployment identity and App run time Identity separate.For DEV -> PROD, the recommended pattern is to deploy the app with DAB from CI/CD, ideally using a d...
- 0 kudos
- 124 Views
- 2 replies
- 1 kudos
Announcement | Operationalizing Genie Ontology in Your Data Stack
Genie Ontology helps Genie One and Genie Agents use business context from governed data, semantic definitions, and existing work assets. The most practical way to improve answer quality is to strengthen that foundation progressively, one business dom...
- 124 Views
- 2 replies
- 1 kudos
- 316 Views
- 1 replies
- 2 kudos
why micro-batching matters so much in Databricks Auto Loader and Structured Streaming
I recently spent some time understanding why micro-batching matters so much in Databricks Auto Loader and Structured Streaming, and it changed the way I look at streaming pipelines.Earlier, I used to think streaming meant processing every record the ...
- 316 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Absolutely agree. The key takeaway for me is that streaming doesn't automatically mean ultra-low latency. Choosing between processingTime and AvailableNow, and lower-latency approaches, should really come down to the business SLA, data arrival patter...
- 2 kudos
- 58 Views
- 1 replies
- 0 kudos
What is a Data Skipping in Delta Lake?
Hi everyone,I’m learning about Delta Lake performance and came across data skipping.I understand that it can help Databricks avoid reading unnecessary data when running queries, but I’d like to understand its purpose more clearly.For example, if an o...
- 58 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@gowri_databrick Data skipping is a built in optimization that uses file-level statistics (minimum values, maximum values and null counts) to automatically skip reading data files that don't contain relevant data for your query. When you write data t...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1380 | |
| 1253 | |
| 1186 | |
| 961 |
Latest from our Blog
Lakeflow Connect: Message Bus Ingestion - Now shipping your logs directly! (Beta)
Take a quick inventory of where your operational data lives right now. There is probably a Kafka cluster carrying your event backbone, or perhaps RabbitMQ capturing messages on a factory floor or bet...
GeoGenie 2: Ask your map a question in plain English
A year ago we put a natural-language layer on a 3D map. This time we rebuilt the whole thing on Lakebase and PostGIS, wired Genie to the same tables, and made the map answer back, drawn region and all...
Custom Visualizations in Databricks AI/BI Dashboards: Build a Dumbbell Chart
Many dashboards eventually answer the same questions: Are we hitting our targets?Where are we falling short? A dumbbell chart answers these questions at a glance. For each category, it plots two valu...
Managing Databricks Genie Agents as Code with Databricks Declarative Automation Bundles (DAB)
Databricks Genie Agents (formerly Genie Spaces) are domain-specific, no-code chat interfaces that let business users ask natural-language questions about their data and get back SQL queries, result ta...
