Databricks Community

sparkplug · 01-27-2025

HiOur costs has suddenly spiked due to logging of a lot of SQL query outputs to DBFS. We haven't made any changes to enable this. How can we disable this feature?

sparkplug · 01-16-2025

I am trying to map out costs for a Shared cluster used in our organization. Since Databricks does not store the sessions in all purpose compute or who accessed the cluster, what are some possible options that I can track which notebooks were attached...

sparkplug · 03-06-2025

Thanks for taking your time to write this detailed blog. It's good to know that Databricks is also thinking the same way.

sparkplug · 02-20-2025

I am also facing the same issue. Is the recommendation to create one A-record for each app, which sounds a bit cumbersome.

sparkplug · 01-27-2025

I don't get any output when running the following, I have the destination set to dbfs . But it was only supposed to be for cluster logs and not for query execution outputs to be stored in DBFS. Any idea if this is expected behavior.spark.conf.get("sp...

sparkplug · 10-28-2024

This api only allows to set permissions based on permission level , which doesn't include changing OWNER. Any suggestions on this particular request.

sparkplug · 10-14-2024

We already have a policy and users are using clusters created with those to run their jobs. Since the policies are not based on job compute but on Power user compute, I am not able to set the job timeout_seconds.

Databricks Community

User Stats

User Activity

Databricks logging of SQL queries to DBFS

How do I track notebooks in all purpose compute?

Re: Understanding Unity Catalog

Re: Databricks App in Azure Databricks with private link cluster (no Public IP)

Re: Databricks logging of SQL queries to DBFS

Re: Notebook and folder owner

Re: Workflow timeout