Set time out or Auto termination for long running query

[NehaR]

Hi ,

We want to set auto termination for long running queries in data bricks adhoc cluster.

I attempted below two approaches in my notebook. Despite my understanding that queries should automatically terminate after one hour, with both the approaches queries exceeded this timeout limit and kept running

Approache 1:

SET spark.databricks.queryWatchdog.enabled=true;

SET spark.databricks.queryWatchdog.timeout=3600;

Approache 2:

SET STATEMENT_TIMEOUT = 3600;

Please suggest what wrong am i doing here?

[JissMathew]

Hi @NehaR  

Apply these settings at the cluster-level configuration in the Databricks UI:

Go to the Cluster Settings.
Add the following Spark configuration:
spark.databricks.queryWatchdog.enabled true

spark.databricks.queryWatchdog.timeout 3600

Restart the cluster to apply the settings.

[PiotrM]

Hey @JissMathew,
I tried those configs on all-purpose clusters, but to no success. Is there any mention in docs that it is supposed to work? 
I know SET STATEMENT_TIMEOUT = 3600 should function only for Warehouses, but I'm looking for a solution that will affect normal clusters. 

[JissMathew]

Hi @PiotrM  For normal all-purpose clusters, you would need an alternative approach to manage query timeouts, as the configuration for STATEMENT_TIMEOUT does not work directly.

Use spark.sql.broadcastTimeout or spark.sql.execution.timeout

spark.conf.set("spark.sql.broadcastTimeout", "3600")
spark.conf.set("spark.sql.execution.timeout", "3600000") 

[NehaR]

Hi @JissMathew ,

Thanks for sharing these settings. I see settings work for sql warehouse clusters and normal all-purpose clusters.

Do we have similar setting option for serverless compute also?