Databricks Community

data_pulse

I'm running ai_decide() in a notebook (SQL cell, serverless compute) and it's now consistently failing with:[INTERNAL_ERROR] Remote session creation failed: INVALID_ARGUMENT: BAD_REQUEST: Endpoint ai_decide is not enabled for batch inference in this ...

data_pulse

Databricks Unity AI Gateway provides a unified way to monitor and understand AI activity through its built-in Workspace AI Gateway Usage Analytics dashboard and the underlying system.ai_gateway.usage system table.Some useful capabilities include:Mode...

data_pulse

Databricks' Governed Tag Propagation (Beta) is an interesting addition to Unity Catalog, helping automatically carry governed tags across downstream tables and views using data lineage.As data moves across medallion layers, this could significantly r...

data_pulse

When teams start looking at Databricks cost, the first instinct is usually to query system.billing.usage.That gives a good view of Databricks consumption, but it is only one part of the overall cost picture..For a more complete FinOps view on Azure, ...

data_pulse · 09-25-2026

Databricks mentions in the DBR 19 release notes that Auto CDF removes write-time overhead and can make MERGE and UPDATE operations about 15% faster on tables queried for changesFor anyone who has moved an existing pipeline from legacy CDF to Auto CDF...

data_pulse

Good points already shared on the nullable → backfill → NOT NULL approach. I'd add a few Lakeflow Connect and Silver recovery considerations, particularly to avoid unnecessary full refreshes.1. Lakeflow Connect: Verify historical data before backfill...

data_pulse

@faruk We handled a similar use case where multiple business teams submitted files with varying structures and frequent template changes. The approach was to separate file collection, schema validation, and Databricks ingestion to make the process mo...

data_pulse

@jimcarre system.ai_gateway.usage provides quite useful coding-agent session metadata, including the agent name/version, session ID, model, token consumption, latency and request status etc.For eg, in my OpenCode validation, I could correlate all suc...

data_pulse

Thanks for the write-up @suranga. I investigated this using managed UC volumes in Databricks. I couldn't reproduce a non-ASCII filename issue, but the tests surfaced a few other ways documents can appear processed successfully while retrieval returns...

data_pulse

@johnb1 Thanks for running these additional tests and the efforts. The results help narrow down the issue considerably.current_date() wasn’t the cause: Removing it eliminated PLAN_NOT_DETERMINISTIC, but the refresh still took 199 minutes. So, while ...

Databricks Community

User Stats

User Activity

ai_decide() failing with Endpoint is not enabled for batch inference in Free Edition

Unity AI Gateway Observability: Models, MCP Tools, and Coding Agents

Governed Tag Propagation (Beta) in Unity Catalog - How It Works and What to Consider

Building an End-to-End FinOps Cost Attribution Model for Azure Databricks

Anyone migrated from legacy CDF to Auto CDF yet?

Re: Schema evolution: new NOT NULL column fails on existing Silver data (Lakeflow Connect source)

Re: How do you collect Excel files from business users into Databricks ?

Re: Unity AI Gateway Observability: Models, MCP Tools, and Coding Agents

Re: Do non-ASCII characters in file names break document indexing over a Unity Catalog volume?

Re: Materialized Views - Strange Difference in Refresh Performance