Data + AI Summit 2026 · San Francisco ️ To Every Summit Attendee – Welcome to Your Community Spotlight! We are FINALLY at Data + AI Summit 2026 and we couldn't be more pumped to be here with YOU! The sessions are 勞, the crowd is electric...
- 8947 Views
- 339 replies
- 34 kudos
a week ago
San Francisco · June 15–18, 2026 Data + AI Summit 2026is almost here. Four days. One conference. And a community that turns up for each other, in person and online. Here's exactly where to find us in San Francisco. → Explore the Summit → View the...
- 630 Views
- 1 replies
- 5 kudos
a week ago
Day 3 Recap · Thursday, June 18, 2026 The Legos ran out.The hugs didn't. Day 3 was a quieter kind of day - keynotes wrapped yesterday, so today belonged to the breakouts. The booth saw slow trickles instead of waves. We ran out of Legos an hour b...
- 166 Views
- 2 replies
- 2 kudos
yesterday
Grow Your Skills and Earn Rewards! Mark your calendar: June 15 – July 6, 2026 Join us for a three-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whethe...
- 79778 Views
- 168 replies
- 87 kudos
05-14-2026
Join Our Business Impact Accelerator Sessions! We are thrilled to invite you to the Business Impact Accelerator, a specialized single-session program designed specifically for business professionals, executives, and non-technical teams. If yo...
- 1033 Views
- 1 replies
- 2 kudos
2 weeks ago
DAIS 2026 · Community Virtual Contest The WinnersAre Here. The votes are counted. The scores are locked. The final day of DAIS 2026 deserves a proper celebration. A few weeks ago, we launched a challenge for our community — build somethin...
- 554 Views
- 5 replies
- 9 kudos
Thursday
Learn how the lakebase architecture decouples compute from storage, so you can build, ship and run agents and apps faster. Databricks has introduced a new category of serverless Postgres database built to integrate seamlessly with your lakehouse sta...
- 524 Views
- 0 replies
- 0 kudos
Monday
Customer service agents often deal with large volumes of questions and requests, which can lead to errors, slower responses and lower customer satisfaction. The Building a Chatbot With Large Language Models (LLMs) Solution Accelerator shows how organ...
- 325 Views
- 0 replies
- 0 kudos
Tuesday
The Weekly Digest • June 8 – June 14 Community Pulse The week before DAIS 2026. Catch up on everything before the big one. This Week's Community Stars Shoutout to the members who kept the community moving this week: @Brahmareddy @balajij8 @S...
- 203 Views
- 0 replies
- 2 kudos
Thursday
Community Activity
- 2669 Views
- 6 replies
- 4 kudos
Resolved! Serverless Compute - pySpark - Any alternative for rdd.getNumPartitions()
Hello Community,We have been trying to migrate our jobs from Classic Compute to Serverless Compute. As part of this process, we face several challenges, and this is one of them.When we read CSV or JSON files with multiLine=true, the load becomes sing...
- 2669 Views
- 6 replies
- 4 kudos
Latest Reply
- 4 kudos
spark_partition_id is the closest and most performant function available as an alternative, and I migrated to use this function. So far, no issues.https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.spark_p...
- 4 kudos
- 1288 Views
- 3 replies
- 0 kudos
Resolved! Serverless Compute - Python - Custom Emails via SMTP (smtplib.SMTP(host_name)) - Any alternative?
Hello Community,We have been trying to migrate our jobs from Classic Compute to Serverless Compute. As part of this process, we face several challenges, and this is one of them.We have several scenarios where we need to send an inline email via Pytho...
- 1288 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
The solution we implemented as an alternative for email sending from Serverless is via the Microsoft Graph API.https://learn.microsoft.com/en-us/graph/api/user-sendmail?view=graph-rest-1.0&tabs=python
- 0 kudos
- 78 Views
- 0 replies
- 0 kudos
200,000 strong and just getting started. My Data and AI Summit 2026
Just back from Data and AI Summit 2026 and I am still buzzing.This was my best summit yet. Moscone was packed. More than 30,000 of us in one place, from over 150 countries, all there for the same reason. To build better with data and AI.The keynotes ...
- 78 Views
- 0 replies
- 0 kudos
- 85 Views
- 0 replies
- 0 kudos
I Took the Databricks Context Engineer Associate Beta Exam. Here Is How I Would Prepare.
The new Databricks Certified Context Engineer Associate beta exam is not a typical certification exam.I took it during Data + AI Summit 2026, and my first reaction was simple: this is a serious exam.I have completed seven Databricks certifications so...
- 85 Views
- 0 replies
- 0 kudos
- 304 Views
- 1 replies
- 2 kudos
DAIS 2026 Speaker Spotlight Series #8 | Fabien Contaminard
DAIS 2026 · Speaker Spotlight A conversationwith Fabien Contaminard On bringing complex event processing into plain SQL with MATCH_RECOGNIZE — and why the humans who own it will own the analysis their agents write. The Session Advanced SQL ...
- 304 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
where can i get the notebooks from Fabien's talks? i'm his biggest fan now
- 2 kudos
- 166 Views
- 2 replies
- 3 kudos
DAIS 2026 | Day 3 Recap: That's a wrap. Empty boxes & full hearts.
Day 3 Recap · Thursday, June 18, 2026 The Legos ran out.The hugs didn't. Day 3 was a quieter kind of day - keynotes wrapped yesterday, so today belonged to the breakouts. The booth saw slow trickles instead of waves. We ran out of Legos an hour b...
- 166 Views
- 2 replies
- 3 kudos
Latest Reply
- 3 kudos
Incredible week with incredible people! Thanks to everyone who contributed on the greater L&E team. I had such a great time, and am feeling pretty beat after all that adrenaline! Looking forward to an even bigger, even better event next year.
- 3 kudos
- 17215 Views
- 4 replies
- 1 kudos
Best way to generate fake data using underlying schema
HiWe are trying to generate fake data to run our tests. For example, we have a pipeline that creates a gold layer fact table form 6 underlying source tables in our silver layer. We want to generate the data in a way that recognises the relationships ...
- 17215 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Nick_Hughes ,I am very late to the party, but I was digging in the internet to find more people discussing a relatable problem for which I am on my way building a definitive solution, and came across your post from 3 years ago. Times have changed...
- 1 kudos
- 66 Views
- 0 replies
- 0 kudos
How a Partitioning Mistake Turned a 12-Minute Databricks Job Into a 2-Hour Nightmare
Hello Databricks Community!I recently published a detailed breakdown on Medium about a real-world optimization nightmare we faced, and I wanted to share the core lessons learned with this group.We had a highly efficient Delta table pipeline handling ...
- 66 Views
- 0 replies
- 0 kudos
- 56 Views
- 0 replies
- 0 kudos
Selective overwrite on Partition and Liquid clustered tables
Hi,I have created 2 identical tables but one is partitioned and the one is a Liquid Clustered with Auto Clustering.I inserted 30M rows x 2 (60M) for two dates , date 1 = 2026-06-01 and date = 2026-06-02 , then I overwrite the date 2026-06-02 with a s...
- 56 Views
- 0 replies
- 0 kudos
- 380 Views
- 7 replies
- 0 kudos
Sizing Tables and delt logs/CDF
Hi,I need to compare the sizes of my delta tables , what's the correct approach ?Table size reported by analyze command ? , but how do I check the delta log size , if I enable CDF .. how do I know the CDF log size(the overhead it adds) ? , kind of l...
- 380 Views
- 7 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @RGSLCA DESCRIBE DETAIL is the best starting point if you're comparing Delta table sizes, but it's important to understand what it reports. The sizeInBytes value represents only the latest active snapshot of the table, not the total storage consum...
- 0 kudos
- 153 Views
- 2 replies
- 0 kudos
Resolved! Get Started with Databricks for Machine Learning
I completed all the steps in the course syllabus and scored a 100 on the final test, but I have not been issued a certificate or badge for the course. Everything is showing as "Completed". What can I do?
- 153 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
@bferreira ,If you want a badge, you had better to prepare more time and then take an examination (that need to paid). And you could read more detail how to get a badge from below list:https://www.databricks.com/learn/training/certificationIt has ano...
- 0 kudos
- 187 Views
- 0 replies
- 0 kudos
Announcement | What’s new in Genie Code at Data + AI Summit 2026
Databricks is expanding Genie Code at Data + AI Summit 2026 with a new full-page workspace for longer-running, multi-step data and ML work. The update is designed to help teams manage more complex development flows across notebooks, files, pipelines,...
- 187 Views
- 0 replies
- 0 kudos
- 367 Views
- 2 replies
- 0 kudos
Resolved! Serverless NCC Private Endpoint ESTABLISHED but traffic routes via eth0 instead of PrivateLink (AWS
Hi community,I've been trying to connect Databricks Serverless to a SQL Serverrunning on an EC2 instance using NCC Private Endpoint, but trafficis not being routed through PrivateLink.## Setup- Databricks Serverless (AWS, us-east-1)- NCC attached to ...
- 367 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Louis thanks,My issue was with the NLB. The Databricks documentation does not specify some of the required settings for this configuration. I found the solution in this post:https://medium.com/databricks-platform-sme/aws-databricks-serverless-private...
- 0 kudos
- 244 Views
- 5 replies
- 0 kudos
Resolved! Manage budgets and cost controls for Genie
In the article Manage budgets and cost controls for Genie at Manage budgets and cost controls for Genie - Azure Databricks | Microsoft Learn it mentions setting the resource type of Unity AI gateway from the dropdown. However when using Account Conso...
- 244 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @AndyRoyle, Per the public docs, the expected flow is that once that preview is enabled, you go to the Budgets page in Account Console, and the budget definition should show a Resource types dropdown where you can choose Unity AI Gateway. You can ...
- 0 kudos
- 2895 Views
- 7 replies
- 4 kudos
Share Your Expertise: Submit Your Blogs and Videos to the Databricks Community!
Are you passionate about sharing your knowledge and insights with the community? We have an exciting opportunity for you! You can now submit your blogs and videos to the Databricks Community for greater exposure and impact. We believe in the power of...
- 2895 Views
- 7 replies
- 4 kudos
Latest Reply
- 4 kudos
Hi Databricks Community,I recently created a short video introducing the basics of Databricks Lakebase in just 7 minutes.In this video, I cover:• What is Databricks Lakebase?• PostgreSQL capabilities• Branching concepts• Lakebase architecture overvie...
- 4 kudos
Announcements
| User | Count |
|---|---|
| 2274 | |
| 1317 | |
| 1228 | |
| 1182 | |
| 959 |
| User | Count |
|---|---|
| 441 | |
| 286 | |
| 220 | |
| 63 | |
| 51 |
Latest from our Blog
Incremental REPLACE WHERE Flows Brings Targeted Refreshes to SDP and DBSQL
Lakeflow Spark Declarative Pipelines (SDP) has long supported two ways to maintain a table: a streaming flow for incremental ingestion and a materialized view for a fully derived result that is always...
One question, two data realities: supervisor agents across Kafka and the lakehouse
The setup every operations analyst knows A procurement lead asks what sounds like a simple question:"Of the orders that came in over the last hour, which ones are from customers who are close to their...
Triggering Databricks Jobs from Celonis Action Flows — A Step-by-Step Guide
Turn process intelligence into automated action: connect Celonis directly to Databricks. Why Connect Celonis Action Flows to Databricks? Celonis shows you where your business processes break down — wh...
Petabyte scale with Zerobus ingest: Download the Code and Ingest the Milky Way
We just published a deep-dive engineering post on Zerobus Ingest that covers how it works and the design choices that made it possible. 1 petabyte. 24 hours. 12 GB/s sustained. 12 million rows per se...
