cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Explore learning resources, share insights, and collaborate with peers to enhance your skills in data engineering, machine learning, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Certifications

Join dynamic discussions on Databricks certifications within the Community. Exchange insights, tips,...

874 Posts

Training offerings

Explore discussions on Databricks training programs and offerings within the Community. Get insights...

198 Posts

Get Started Discussions

Start your journey with Databricks by joining discussions on getting started guides, tutorials, and ...

2709 Posts

Activity in Community Discussions

tarunnagar
by > Contributor
  • 51 Views
  • 2 replies
  • 2 kudos

Using Databricks for Real-Time App Data

I’m exploring how to handle real-time data for an application and I keep seeing Databricks recommended as a strong option — especially with its support for streaming pipelines, Delta Live Tables, and integrations with various event sources. That said...

  • 51 Views
  • 2 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

Real-time structured streaming is offering ms latency. Of course, if you want to save in the Delta format, it can take a few seconds and incur high costs due to frequent write operations to storage. The best solution in structured streaming is an eve...

  • 2 kudos
1 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 24 Views
  • 0 replies
  • 2 kudos

Databricks Advent Calendar 2025 #9

Tags, whether manually assigned or automatically assigned by the “data classification” service, can be protected using policies. Column masking can automatically mask columns with a given tag for all except some with elevated access.

2025_9.png
  • 24 Views
  • 0 replies
  • 2 kudos
Renounce3295
by > New Contributor II
  • 6774 Views
  • 8 replies
  • 4 kudos

Not able to serve or interact with LLMs in Databricks free

Hi there,Just testing the new Databricks free edition. Was trying to play around with LLMs, but I', not able to create serving endpoints with foundational model entities, interact with pay-per-token foundational model APIs or use them in Databricks a...

  • 6774 Views
  • 8 replies
  • 4 kudos
Latest Reply
chaowu2016
New Contributor
  • 4 kudos

I got the same error using the free version.

  • 4 kudos
7 More Replies
tarunnagar
by > Contributor
  • 117 Views
  • 3 replies
  • 0 kudos

How to Connect Databricks with Web and Mobile Apps

Hi everyone,I’m exploring ways to leverage Databricks for building data-driven web and mobile applications and wanted to get some insights from this community. Databricks is great for processing large datasets, running analytics, and building machine...

  • 117 Views
  • 3 replies
  • 0 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 0 kudos

Check Databricks Apps - you pass databricks resources and then use databricks-sdk to interact with them.

  • 0 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 41 Views
  • 0 replies
  • 1 kudos

Databricks Advent Calendar 2025 #8

Data classification automatically tags Unity Catalog tables and is now available in system tables as well.

2025_8.png
  • 41 Views
  • 0 replies
  • 1 kudos
Eswariy
by > New Contributor
  • 118 Views
  • 2 replies
  • 0 kudos

Request to reschedule Databricks Certified Generative AI Engineer Associate

I am writing to request assistance with rescheduling my exam for the Databricks Certified Generative AI Engineer Associate certification. My exam was originally scheduled for Today at 7:30 PM. However, when I attempted to access the exam, the Webasse...

  • 118 Views
  • 2 replies
  • 0 kudos
Latest Reply
cert-ops
Databricks Employee
  • 0 kudos

Hello @Eswariy ,  Sorry to hear you missed your exam window. Please file a ticket with our support team so they can review the case and determine next steps. Please note that we cannot provide support via community. Thanks & Regards,@cert-ops

  • 0 kudos
1 More Replies
saikrishna08
by > New Contributor
  • 91 Views
  • 1 replies
  • 1 kudos

Data Engineer Professional Certificate Exam Discount Voucher

HiI am planning to take Databricks Certified Data Engineer Professional Exam. The fee is highly unaffordable for me. I am looking for discount vouchers. Please let me know if there is a way to get discount on the exam.Thank you!RegardsKrishna

  • 91 Views
  • 1 replies
  • 1 kudos
Latest Reply
GaweL
New Contributor
  • 1 kudos

Hi @saikrishna08,As far as i know Databricks usually offers exam discount vouchers during their quarterly Learning Festivals and sometimes through partner organizations.Check out this thread for more information about this topic.https://community.dat...

  • 1 kudos
Idrisskh
by > New Contributor II
  • 147 Views
  • 3 replies
  • 2 kudos

Gemini video input

Hello,Is it possible to use a video as an input for Google Gemini 2.5 Pro within Databricks?Thank you.

  • 147 Views
  • 3 replies
  • 2 kudos
Latest Reply
Idrisskh
New Contributor II
  • 2 kudos

Do you know when it will be available?

  • 2 kudos
2 More Replies
boitumelodikoko
by > Valued Contributor
  • 612 Views
  • 1 replies
  • 3 kudos

Data Engineering Lessons

Getting into the data space can feel overwhelming, with so many tools, terms, and technologies. But after years inExpect failure. Design for it.Jobs will fail. The data will be late. Build systems that can recover gracefully, and continually monitor ...

  • 612 Views
  • 1 replies
  • 3 kudos
Latest Reply
Gecofer
Contributor
  • 3 kudos

Hi @boitumelodikoko A few more principles I always share with people entering the data space:Observability is non-negotiable.If you can’t see what your pipelines are doing, you can’t fix what breaks.Good logging, metrics, and alerts save countless ho...

  • 3 kudos
Poorva21
by > New Contributor
  • 105 Views
  • 1 replies
  • 2 kudos

How realistic is truly end-to-end LLMOps on Databricks?

Databricks is positioning the platform as a full stack for LLM development — from data ingestion → feature/embedding pipelines → fine-tuning (Mosaic AI) → evaluation → deployment (Model Serving) → monitoring (Lakehouse Monitoring).I’m curious about r...

  • 105 Views
  • 1 replies
  • 2 kudos
Latest Reply
Gecofer
Contributor
  • 2 kudos

Hi @Poorva21 In several projects I’ve seen and worked on, Databricks gets you very close to a full end-to-end LLMOps platform, but not completely. It realistically covers most of the lifecycle, but in real production setups you still complement it wi...

  • 2 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 74 Views
  • 0 replies
  • 2 kudos

Databricks Advent Calendar 2025 #7

Imagine all a data engineer or analyst needs to do to read from a REST API is use spark.read(), no direct request calls, no manual JSON parsing - just spark .read. That’s the power of a custom Spark Data Source. Soon, we will see a surge of open-sour...

2025_7.png 2025_7.png
  • 74 Views
  • 0 replies
  • 2 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 98 Views
  • 0 replies
  • 2 kudos

Databricks Advent Calendar 2025 #6

DBX is one of the most crucial projects of dblabs this year, and we can expect that more and more great checks from it will be supported natively in databricks. More about dbx on https://databrickslabs.github.io/dqx/

2025_6.png
  • 98 Views
  • 0 replies
  • 2 kudos
n1399
by > New Contributor II
  • 1071 Views
  • 2 replies
  • 0 kudos

On Demand Pool Configuration & Policy definition

I'm using Job cluster and created compute policies for library management and now I'm trying to use pools in databricks. I'm getting error like this : Cluster validation error: Validation failed for azure_attributes.spot_bid_max_price from pool, the ...

  • 1071 Views
  • 2 replies
  • 0 kudos
Latest Reply
Poorva21
New Contributor
  • 0 kudos

This error occurs because instance pools require a concrete spot bid max price value, even if the cluster policy marks it as unlimited. Set an explicit value (e.g., 100) directly in the instance pool configuration, or switch the pool to on-demand nod...

  • 0 kudos
1 More Replies
mrstevegross
by > Contributor III
  • 5111 Views
  • 2 replies
  • 2 kudos

How to resolve "cannot import name 'Iterable' from 'collections'" error?

I'm running a DBR/Spark job using a container. I've set docker_image.url to `docker.io/databricksruntime/standard:13.3-LTS`, as well as the Spark env var `DATABRICKS_RUNTIME_VERSION=13.3`. At runtime, however, I'm encountering this error: ImportError...

  • 5111 Views
  • 2 replies
  • 2 kudos
Latest Reply
Poorva21
New Contributor
  • 2 kudos

Go to Compute → Your Cluster / Job ComputeChange Databricks Runtime to:Databricks Runtime 13.3 LTSRe-run your job with the same container.

  • 2 kudos
1 More Replies
steveKris
by > New Contributor
  • 222 Views
  • 6 replies
  • 4 kudos

Extract all users from Databricks Groups

Hey everyone,we are trying to get an overview of all users that we have in our databricks groups. We have tried to do so with the REST API as well as the SQL-queries (with normal developer accounts as well as workspace administrator accounts). The pr...

  • 222 Views
  • 6 replies
  • 4 kudos
Latest Reply
Poorva21
New Contributor
  • 4 kudos

Use the Databricks SQL system users tableSELECT * FROM system.usersOnly shows fully provisioned users Users pending invitation may not appear.

  • 4 kudos
5 More Replies