JissMathew
Valued Contributor

Hi @PiotrM  For normal all-purpose clusters, you would need an alternative approach to manage query timeouts, as the configuration for STATEMENT_TIMEOUT does not work directly.

Use spark.sql.broadcastTimeout or spark.sql.execution.timeout

spark.conf.set("spark.sql.broadcastTimeout", "3600")
spark.conf.set("spark.sql.execution.timeout", "3600000") 

Jiss Mathew
India .