Databricks Community

Yogesh_Verma_

Real-Time Mode in Spark StreamingApache Spark™ Structured Streaming has been the backbone of mission-critical pipelines for years — from ETL to near real-time analytics and machine learning.Now, Databricks has introduced something game-changing: Real...

Yogesh_Verma_ · 08-28-2025

If the UI keeps failing with that vague error, the CLI approach suggested above is the best next step, since it usually gives a clearer error message. Also make sure that:The service principal you’re using to create the scope has Key Vault Administra...

Yogesh_Verma_ · 08-28-2025

You’re running into this because DAB treats the YAML definition as the source of truth — so every time you redeploy, it will reset the job state (including the paused/running status) back to what’s defined in the file. Unfortunately, there isn’t curr...

Yogesh_Verma_ · 08-28-2025

Your YAML is valid, but the reason the libraries are not being installed is because base_environment_path is not supported for serverless compute. Serverless jobs use a fully managed environment and you can’t override it with a custom base environmen...

Yogesh_Verma_ · 08-28-2025

Yes, it’s possible to include the Databricks job name in your Brooklyn audit tables, but it won’t happen automatically. Right now, only the job run ID is being logged, so you’d need to extend your audit logic a bit. One common approach is to pass the...

Yogesh_Verma_ · 08-28-2025

The endpoint is working since you’re receiving responses, but the inference table isn’t capturing any records. This points to a logging/compatibility issue rather than request handling. Please confirm inference logging is enabled, verify permissions,...

Databricks Community

User Stats

User Activity

Real-Time Mode in Apache Spark Structured Streaming

Re: Not able to creare Secret scope in Azure databricks

Re: Excluding job update from DAB .yml deployment

Re: Serverless base env setup in Databricks Asset Bundle (DAB)

Re: Databricks job calling DBT - persist job name

Re: Inference table not working for Gemma 3 12b