Set time out or Auto termination for long running query

[NehaR]

Hi ,

We want to set auto termination for long running queries in data bricks adhoc cluster.

I attempted below two approaches in my notebook. Despite my understanding that queries should automatically terminate after one hour, with both the approaches queries exceeded this timeout limit and kept running

Approache 1:

SET spark.databricks.queryWatchdog.enabled=true;

SET spark.databricks.queryWatchdog.timeout=3600;

Approache 2:

SET STATEMENT_TIMEOUT = 3600;

Please suggest what wrong am i doing here?

[JissMathew]

Hi @NehaR  

Apply these settings at the cluster-level configuration in the Databricks UI:

Go to the Cluster Settings.
Add the following Spark configuration:
spark.databricks.queryWatchdog.enabled true

spark.databricks.queryWatchdog.timeout 3600

Restart the cluster to apply the settings.