Databricks Free Edition is the new home for personal learning and exploration on Databricks. It’s perpetually free and built on modern Databricks - the same Data Intelligence Platform used by professionals. Free Edition lets you learn professional da...
- 1346 Views
- 1 replies
- 4 kudos
2 weeks ago
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 1784 Views
- 4 replies
- 6 kudos
2 weeks ago
Your voice matters to us. We are planning the 2026 Data + AI Summit, and we’d love your input on what would make the experience even more valuable for you. Take a few minutes to share your feedback through our quick survey — your insights directly in...
- 800 Views
- 3 replies
- 5 kudos
2 weeks ago
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 3341 Views
- 2 replies
- 9 kudos
10-02-2025
Our Champion program has always celebrated the customers who go above and beyond to engage, help others, and uplift the Community. Recently, we have seen remarkable participation from Bricksters as well—and their impact deserves recognition too. Begi...
- 1488 Views
- 7 replies
- 14 kudos
11-21-2025
Learning doesn’t pause, and neither does the impact this Community continues to create!Across threads and time zones, the knowledge kept moving. Catch up on the highlights Voices Shaping the Week Featuring the voices that brought clarity, ...
- 430 Views
- 1 replies
- 1 kudos
Tuesday
Community Activity
- 5 Views
- 1 replies
- 0 kudos
Need Guidance for Databricks-Generative-AI-Engineer-Associate Exam
Hi Everyone,I’m preparing for the Databricks-Generative-AI-Engineer-Associate exam and looking for some guidance from experienced candidates. I want to understand the exam pattern, important topics, and the best ways to practice for success.I’ve been...
- 5 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, Here are some example questions for you to practise on, I used a LLM to generate them and could use a similar approach to generate some more. I'd also recommend looking at some of the exam prep content on udemy, there are a few courses on there w...
- 0 kudos
- 19 Views
- 2 replies
- 0 kudos
Service Principal
Can two service principal have same name,but unique id's ?
- 19 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @SP_6721 ,do you know how and why it behaves like this ?
- 0 kudos
- 24 Views
- 1 replies
- 0 kudos
Getting error when running databricks deploy bundle command
HI all,I am trying to implement MLOps project using https://github.com/databricks/mlops-stacks repo.I have created azure databricks with Premium (+ Role-based access controls) (Click to change) and following bundle creation and deploy using uRL: http...
- 24 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, I think this may be a duplicate of another question, but posting the same answer here for transparency: Hi, first things to check is that you have the correct permissions on the user or service principal you're running the job with, the user need...
- 0 kudos
- 116 Views
- 1 replies
- 4 kudos
Designing Reliable Stream–Stream Joins with Watermarks in Databricks
Stream–stream joins are one of the most powerful features in Databricks Structured Streaming – and also one of the easiest to misconfigure. As soon as you move from simple append-only pipelines to real-time correlations across multiple streams (order...
- 116 Views
- 1 replies
- 4 kudos
Latest Reply
- 4 kudos
Interesting and very Insightful @Divaker_Soni.
- 4 kudos
- 66 Views
- 1 replies
- 0 kudos
Getting error when running databricks deploy bundle command
HI all,I am trying to implement MLOps project using https://github.com/databricks/mlops-stacks repo.I have created azure databricks with Premium (+ Role-based access controls) (Click to change) and following bundle creation and deploy using uRL: http...
- 66 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, first things to check is that you have the correct permissions on the user or service principal you're running the job with, the user needs to have workspace access and cluster creation access toggled on. Next you need to check you have a metast...
- 0 kudos
- 222 Views
- 5 replies
- 1 kudos
Azure Content Understanding Equivalent
Hi all,I am exploring Databricks services or components that could be considered equivalent to Azure Document Intelligence and Azure Content Understanding.Our customer works with dozens of Excel and PDF files. These files follow multiple template typ...
- 222 Views
- 5 replies
- 1 kudos
Latest Reply
- 1 kudos
HI, agent bricks which information extraction is part of is https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ which is only available in US regions at the moment. The AI_parse_document function is available in the EU on AWS but not azur...
- 1 kudos
- 118 Views
- 4 replies
- 4 kudos
Is anyone getting up and working ? Federating Snowflake-managed Iceberg tables into Azure Databricks
I'm federating Snowflake-managed Iceberg tables into Azure Databricks Unity Catalog to query the same data from both platforms without copying it. I am getting weird error message when query table from Databricks and i have tried to put all nicely in...
- 118 Views
- 4 replies
- 4 kudos
Latest Reply
- 4 kudos
Thanks Hubert. I did check the Iceberg metadata location and Databricks can list the files, but the issue is that Snowflake’s Iceberg metadata.json contains paths like abfss://…@<acct>.blob.core.windows.net/..., and on UC Serverless Databricks then t...
- 4 kudos
- 1112 Views
- 6 replies
- 4 kudos
Docker tab missing in create compute
I am running databricks premium and looking to create a compute running conda. It seems that the best way to do this is to boot the compute from a docker image. However, in the ```create_compute > advanced``` I cannot see the the docker option nor ca...
- 1112 Views
- 6 replies
- 4 kudos
Latest Reply
- 4 kudos
Hi @Askenm In Databricks Premium, the Docker option for custom images is not available on all compute types and is not controlled by user level permissions. Custom Docker images are only supported on Databricks clusters that use the legacy VM based c...
- 4 kudos
- 8866 Views
- 4 replies
- 2 kudos
Optimizing Costs in Databricks by Dynamically Choosing Cluster Sizes
Databricks is a popular unified data analytics platform known for its powerful data processing capabilities and seamless integration with Apache Spark. However, managing and optimizing costs in Databricks can be challenging, especially when it comes ...
- 8866 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
@Second Reply You’re right just printing out selected_pool isn’t enough to actually leverage dynamic cluster sizing at runtime. In practice, the value of selected_pool would feed directly into your Databricks cluster creation API or workflow automati...
- 2 kudos
- 126 Views
- 3 replies
- 2 kudos
Resolved! Calling a function with parameters via Spark ODBC driver
Hi All,I am having an issue with calling a Databricks SQL user defined function with parameters from my client application using the Spark ODBC driver.I have been able to execute a straight SQL statement using parameters e.g. SELECT * FROM Customer W...
- 126 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @CHorton The Databricks SQL engine does not support positional (?) parameters inside SQL UDF calls. When Spark SQL parses GetCustomerData(?), the parameter is unresolved at analysis time, so you get [UNBOUND_SQL_PARAMETER]. This is not an ODBC bu...
- 2 kudos
- 12646 Views
- 2 replies
- 3 kudos
How to change the number of executors instances in databricks
I know that Databricks runs one executor per worker node. Can i change the no.of.exectors by adding params (spark.executor.instances) in the cluster advance option? and also can i pass this parameter when i schedule a task, so that particular task wi...
- 12646 Views
- 2 replies
- 3 kudos
Latest Reply
- 3 kudos
In Databricks, the executor model is largely managed by the platform itself. On Databricks clusters, each worker node typically runs a single Spark executor, and this behavior is intentional.
- 3 kudos
- 97 Views
- 3 replies
- 3 kudos
Resolved! Spark verison errors in "Build an ETL pipeline with Lakeflow Spark Declarative Pipelines"
I'm trying to define a job for a pipeline using the Asset Bundle Python SDK. I created the pipeline first (using the SDK) and i'm now trying to add the Job. The DAB validates and deploys successfully, but when I run the Job i get an error: UNAUTHORIZ...
- 97 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
This happens because the job is not actually linked to the deployed pipeline and the pipeline id is None at runtime. When using Asset Bundles, the pipeline id is only resolved after deployment, so referencing my_pipeline.id in code does not work. Ins...
- 3 kudos
- 167 Views
- 2 replies
- 6 kudos
Meta Ads is now a native data source in Databricks Lakeflow Connect
Meta Ads is now a native data source in DatabricksDatabricks just announced a Meta Ads connector (Beta) powered by Lakeflow Connect, making it easy to ingest advertising data directly into Databricks—no custom APIs, no CSV exports, no brittle scripts...
- 167 Views
- 2 replies
- 6 kudos
Latest Reply
- 6 kudos
Have to test it. Confluence Lakeflow Connect is amazing, so I suspect that one also!
- 6 kudos
- 56 Views
- 0 replies
- 1 kudos
Lakebase Use Cases
I am still amazed by Lakebase and all the possible use cases that we can achieve. Integration of Lakebase with Lakehouse is the innovation of the year. Please read my blog posts to see why it is the best of two worlds. #databricks Read here:- https:/...
- 56 Views
- 0 replies
- 1 kudos
- 148 Views
- 3 replies
- 1 kudos
Resolved! Iceberg interoperability between Databricks and external catalogs
I would like to understand the current approach for Iceberg interoperability in Databricks. Databricks supports Iceberg using Unity Catalog, but many teams also use Iceberg tables managed outside Databricks. Are there recommended patterns today for s...
- 148 Views
- 3 replies
- 1 kudos
Announcements
| User | Count |
|---|---|
| 2207 | |
| 1218 | |
| 1062 | |
| 950 | |
| 888 |
Latest from our Blog
[PARTNER BLOG] Zerobus Ingest on Databricks
Introduction TL;DR ZeroBus Ingest is a serverless, Kafka-free ingestion service in Databricks that allows applications and IoT devices to stream data directly into Delta Lake with low latency and mini...
[PARTNER BLOG] Beyond ADLS Limits: File Arrival Triggers for File Updates
Beyond ADLS Limitations: Making File Arrival Triggers Work for Existing File Updates Using a Flag File MechanismThe Flag File MechanismThe Root Problem: Triggers Only Work on “Create”, Not “Modify” Ev...
AI Newsletter Series (Part 1): Parsing Enterprise PDFs Into AI-Ready Data
Enterprise Account macro trends, strategy doc, account and project updates often end up in PDF format. Meanwhile, usage metrics and account-level signals—such as active users, DBU consumption, and use...
Fine-Grained Access Control in Delta Sharing: Dynamic Views vs Partition Filtering
The Challenge: Sharing Data While Maintaining Privacy Boundaries Imagine you're a global retail company with customer order data spanning multiple regions—North America, Europe, and Asia Pacific. You ...
