Canceling long running on UC-enabled all-purpose clusters
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-02-2025 08:50 AM
Hey,
as in the subject. Is it possible to set timeout for long running queries on all-purpose clusters that are UC enabled?
I know there is such setting for SQL Warehouses and Workflows, but I was unable to find one for all-purpose clusters. The issue we're facing are exploratory queries from those clusters that query SQL Server DBs through Lakehouse Federation. It occurred that those queries got stuck and clusters were finally terminated with the nightly clean-up.
I tried thing like spark.task.reaper.killTimeout, but it seems like UC clusters won't accept it.