Databricks Community

jar

Hi.We have a range of scheduled jobs and _one_ continuous job all defined in .yml and deployed with DAB. The continuous job is paused per default and we use a scheduled job of a notebook to pause and unpause it so that it only runs during business ho...

jar · 07-28-2025

If using spark.conf.set(<variable_name>, <variable_value>), or just referring a widget value directly, in a Python cell and then referring to it in a SQL cell with ${variable_name} one gets the warning: "SQL query contains a dollar sign parameter, $p...

jar · 07-17-2025

Hi.I can see in our Azure cost analysis tool that a not insignificant part of our costs come from the managed Databricks RG deployed with the workspace, and that it relates particularly to VMs (so compute, I assume?) and storage, the later which, tho...

jar · 07-13-2025

Hey.I read at one-point that there would be a cheaper, less-performant version of the Databricks serverless SQL DW but can't find any information on this. Was I dreaming or will it soon be possible with a more cost-effective option for less-intensive...

jar · 07-07-2025

Hello.Is it possible to disable Photon for a serverless SQL DW? If yes, how?Best,Johan.

jar · 07-28-2025

Frustrating indeed. Thank you, @lingareddy_Alva

jar · 07-21-2025

Thank you all for your replies. The issue is not getting an overview of costs - I already have that from the Cost Management Export function in Azure, and by using the system.billing tables in Databricks. The issue is understanding the relation betwe...

jar · 07-20-2025

Hello @_G_ .That would be Databricks costs. I am trying to tie those to Databricks related Azure costs, of which you cannot find any data in the system tables. VMs in the Databricks managed RG seem to be automatically provisioned by Databricks to sup...

jar · 07-14-2025

Hello @szymon_dybczak.Thank you for clarifying. The most cost-effective serverless SQL option for PBI Direct Query is then the 2x-small serverless DW - the different performance modes only applies to notebooks and jobs?

jar · 05-18-2025

So the hybrid solution turned out to be the smartest, at least for my case, with a simple time check that defines the connection information according to whether the query is sent outside business hours or not.

Databricks Community

User Stats

User Activity

Excluding job update from DAB .yml deployment

Use of Python variable in SQL cell

Understanding infra costs of Databricks compute

Databricks serverless SQL cost-effective.. when?

Disable Photon for serverless SQL DW

Re: Use of Python variable in SQL cell

Re: Understanding infra costs of Databricks compute

Re: Understanding infra costs of Databricks compute

Re: Databricks serverless SQL cost-effective.. when?

Re: Define time interval for when a cluster can be active