The Genie has spoken And the winners of the Genie-Powered App Challenge are… A few weeks ago, we threw down a challenge: build something incredible with Databricks Apps and Genie Agent. And wow, you delivered. From smart, real-world prob...
- 1321 Views
- 10 replies
- 8 kudos
Thursday
Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, September 17, 2026! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with d...
- 608 Views
- 2 replies
- 3 kudos
a week ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 897 Views
- 0 replies
- 3 kudos
2 weeks ago
Databricks Advanced Learning Festival September 16 – October 14, 2026 Join us for a four-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whether you are...
- 85634 Views
- 128 replies
- 50 kudos
4 weeks ago
Virtual Event Time: AMER Sept 23 / 9 AM PTEMEA Sept 24 / 9 AM BST / 10 AM CESTAPJ Sept 24 / 9:30 AM IST / 12 PM SGT / 1 PM JST/KST / 2 PM AEST (Japanese and Korean captions available) Register for the Workshop Note: Marking RSVP on this Community ...
- 740 Views
- 0 replies
- 2 kudos
2 weeks ago
Databricks EMEA Learning Festival Hands On Virtual Training 29–30 September 2026 · 09:00 BST / 10:00 CEST · Fully virtual · Free to join Ready to accelerate your journey on Data & AI? Join us for the EMEA Learning Festival. A two-day, hands-on vir...
- 2999 Views
- 0 replies
- 3 kudos
08-11-2026
Community Champion • Our Community Champion Program celebrates members who consistently contribute their expertise, support fellow practitioners, and help shape a stronger and more collaborative Databricks Community. Every month, we recognize individ...
- 385 Views
- 6 replies
- 6 kudos
Monday
Building a customer 360 requires connecting customer records across disparate data sets and establishing common customer identities. The Customer Entity Resolution Solution Accelerator shows how to build that foundation by translating customer attrib...
- 98 Views
- 0 replies
- 2 kudos
Tuesday
The Weekly Digest • August 31 – September 6 Community Pulse Your weekly pulse: new articles, real answers, and the discussions worth watching. This Week's Community Stars This week's difference-makers – and a shout-out to the new members ear...
- 478 Views
- 1 replies
- 2 kudos
a week ago
Community Activity
- 203 Views
- 4 replies
- 4 kudos
Resolved! How do you test whether a retriever stops too early?
Hello everyone,Databricks’ Adaptive Instructed-Retriever announcement describes a model that learns when to stop searching. How would you test whether it stops after finding a general policy but before finding the exception that changes the answer?I...
- 203 Views
- 4 replies
- 4 kudos
Latest Reply
- 4 kudos
This is such an interesting problem! I really like the idea of testing whether extra search recovers missing evidence. Curious to see what metrics others use to balance latency and answer quality.
- 4 kudos
- 5 Views
- 0 replies
- 2 kudos
🌟 Community Pulse: Your Weekly Roundup! September 07 – 13, 2026
The Weekly Digest • September 7 – September 13 Community Pulse A new week, a fresh wave of articles, answers, and discussions. Here's your weekly catch-up. This Week's Community Stars The members who showed up and made a difference this week ...
- 5 Views
- 0 replies
- 2 kudos
- 142 Views
- 6 replies
- 7 kudos
Resolved! Photon enabled but a large share of the plan is falling back, cost up and runtime flat
Hi everyone,Trying to work out whether this is expected or whether I have misconfigured something.We enabled Photon on a job cluster running a nightly aggregation over roughly 2TB. The expectation was the usual improvement. What we got instead was ru...
- 142 Views
- 6 replies
- 7 kudos
Latest Reply
- 7 kudos
Great questions! Would love to hear from someone who’s dealt with Photon fallbacks, especially around Python UDFs. Curious to know if rewriting them actually made a noticeable difference in runtime and cost.
- 7 kudos
- 21197 Views
- 11 replies
- 2 kudos
Resolved! Performance enhancement while writing dataframes into Parquet tables
Hi,I am trying to write the contents of a dataframe into a parquet table using the command below.df.write.mode("overwrite").format("parquet").saveAsTable("sample_parquet_table")The dataframe contains an extract from one of our source systems, which h...
- 21197 Views
- 11 replies
- 2 kudos
Latest Reply
- 2 kudos
Writing an effective essay takes more than putting ideas on paper; it requires solid research, logical structure, clear arguments, and careful proofreading. Students working on challenging assignments may find essay writing help useful for organizing...
- 2 kudos
- 12 Views
- 0 replies
- 0 kudos
using remote_query with SQL Serverless Warehouse
Hello, Im trying to set up remote_query() as an option for our developers to use when querying our DB2 server. Unfortunately the connection keeps timing out. The workspace is in a VPC in AWS and the interactive clusters can successfully use the JDBC ...
- 12 Views
- 0 replies
- 0 kudos
- 64 Views
- 2 replies
- 0 kudos
Databricks Free Edition GitHub push fails with "Resolver error"
Hi,I'm using Databricks Free Edition and connected a GitHub repository through the Databricks GitHub integration. I can clone/connect to the repository and create/edit files, but when I try to Commit & Push, Databricks returns:Error pushing changesRe...
- 64 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
@wilsonz3 I tested this in Databricks Free Edition using the Databricks GitHub App and was able to clone, commit, push and pull successfully including a direct git push from the Git folder. So this doesn’t appear to be a general Free Edition limitati...
- 0 kudos
- 96 Views
- 2 replies
- 1 kudos
Error with Databricks ODBC Driver OAuth 2.0
Hi,I am facing some issues while configuring Databricks ODBC driver for Tableau connection purposes. After setting up all the parameters and settings I get the following error when I use OAuth 2.0 mechanism with Browser Based Authorization Code. Has ...
- 96 Views
- 2 replies
- 1 kudos
- 18 Views
- 0 replies
- 0 kudos
Learn Databricks Apache Spark Declarative Pipelines | From Associate to Professional
Databricks Training & Certifications Learn Databricks Apache Spark Declarative Pipelines Declare what you want. Let the engine build and run the pipeline for you. Declarative ETL ◆ Streaming tables & MVs ✓ Data quality With Apache Spark™ Decl...
- 18 Views
- 0 replies
- 0 kudos
- 588 Views
- 2 replies
- 2 kudos
Actually understanding Unity Catalog Managed Tables
The “why” worth knowing Choosing external tables is usually about flexibility: you keep dedicated storage, maintain raw files that multiple engines can query directly, and control your own partitioning. But control is also a constraint. To guarantee ...
- 588 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Thanks @ivanvyd! Really good point! Infrequent readers can indeed fall outside the observation window, but what actually matters is not how often it runs but whether it's a verified reader. If you're running this report from Databricks on the latest...
- 2 kudos
- 70 Views
- 2 replies
- 0 kudos
help me how to fix
"Triggering new runs for organization 7474657040075149 is currently disabled temporarily." , Why did my job get blocked while running? You clearly said it could run continuously, so why was it blocked?
- 70 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @FastFoodBro The Databricks Community and Community/Free tier workspaces have hard limits on total cluster compute time per organization ID, measured in hours or days. Once you hit your continuous job limit, the next runs are blocked from launchin...
- 0 kudos
- 24 Views
- 0 replies
- 0 kudos
DLT Pipeline - Overwrite except for one Append table
Hi all,I am a junior engineer and I am working on a use case involving DLT pipelines that read from a csv source at regular intervals and populate tables based upon processing of this. In most cases, I'd like the table contents to be overwritten with...
- 24 Views
- 0 replies
- 0 kudos
- 24 Views
- 0 replies
- 0 kudos
Building Custom Agents on Databricks:LangGraph, Atlan-Grounded Routing, and Per-Run Cost with MLflow
The problem: rethinking routing in multi-agent systems on DatabricksDatabricks native supervisor agent pattern is a solid default for orchestrating multiple Genie spaces and tools but it comes with a routing constraint worth examining. The superviso...
- 24 Views
- 0 replies
- 0 kudos
- 230 Views
- 7 replies
- 4 kudos
Compatibility Mode for AWS Redshift
Hello,I have several tables loaded from S3 using Lakeflow pipelines which generates streaming table and materialized views. Want to be able to read these tables in AWS redshift.Based on the Databricks documentation for compatibility mode:Databricks C...
- 230 Views
- 7 replies
- 4 kudos
Latest Reply
- 4 kudos
Thanks @ivanvyd , from describe extended table, i can see metadata location as s3-compat, detailed table information has streaming table, location s3-uc. In theory, the databricks unity catalog connector with AWS should read metadata from the s3-comp...
- 4 kudos
- 577 Views
- 12 replies
- 1 kudos
Can I run jobs continuously without interruption on a permanent free (Community Edition) account?
Chào mọi người,Tôi hiện đang sử dụng Databricks Community Edition (tài khoản miễn phí) và muốn hỏi về việc chạy các tác vụ theo lịch trình/liên tục trên đó.Cụ thể: 1. Liệu có thể chạy một tác vụ liên tục (ví dụ: tác vụ xử lý dữ liệu trực tuyến hoặc t...
- 577 Views
- 12 replies
- 1 kudos
Latest Reply
- 1 kudos
@FastFoodBro You can use Databricks Free account for personal use only (personal use, learning, experimentation - not for production or commercial use). You can upgrade to a paid plan to access full platform features as you will face interruptions in...
- 1 kudos
- 115 Views
- 4 replies
- 5 kudos
Resolved! At what data size do you stop reaching for Spark?
Hi everyone,A question I keep having with my team and I would like to hear how others think about it.A lot of the jobs we run are not big. Plenty of our pipelines process a few gigabytes, some considerably less. We run them on Spark because that is w...
- 115 Views
- 4 replies
- 5 kudos
Latest Reply
- 5 kudos
Is there a size threshold?Most teams that have thought hard about this land somewhere around "if a job's working set fits comfortably in memory on one large-ish node and doesn't need to scale unpredictably, Spark's distributed machinery is pure overh...
- 5 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1420 | |
| 1253 | |
| 1187 | |
| 961 |
Latest from our Blog
Beyond the Dashboard: How Transferz Built a Truly Data-Driven Company Culture With AI/BI Genie
Transferz, a fast-growing travel tech platform, eliminated its data team bottleneck by deploying AI/BI Genie — achieving 89,5% adoption across business users achieving 60% operational improvements.Th...
5 Ways to Enhance Databricks AI/BI Dashboard Tables with HTML
Organizations use AI/BI dashboard tables to review accounts, catch problems, and decide what to do next. Yet a plain table can hold all the right data and still be hard to read. One way to improve the...
Tutorial: Transform your Lakeflow Connect ad data into visual and conversational analytics
Introduction Traditionally, unifying requires building brittle custom integrations for every single source. Lakeflow Connect resolves this with native, fully managed connectors for a full suite of mar...
Meet SDP Rewind: An undo button for your ETL pipelines
A bad deployment slips into your pipeline on Friday, and by Monday, it has been writing incorrect data for three days, with no clean way to roll it back. You can restore one table, but a pipeline is a...
