Databricks Community

We have a few serverless dlt pipelines that we want to optimize for cost as we are ok with an increased latency. Where can I change the pipeline to run on cost optimized mode. I dont see this option in UI or API

Looking at the Databricks Java SDK for Pipeline events, I see that the Rest API returns a details field that has the same information as event log details. But this is not surfaced in SDK, should be a small change to add it. Is that something which c...

We accidentally disabled billing system table schema and now getting this error on re-enabling it:Error: billing system schema can only be enabled by Databricks.How can we resolve the same. We have not purchased any of the support contracts and hence...

I am using the system.access.table_lineage table  to figure out the tables accessed by sql queries and the corresponding SQL queries. However I am noticing this table missing data or values very often.For eg for sql queries executed by our DBT jobs, ...

I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been mor...