08-08-2025 10:43 PM
This looks like a misconfigured Query Watchdog, specifically the below config:
spark.conf.get("spark.databricks.queryWatchdog.outputRatioThreshold")
Please check the value of this config - it is 1000 by default. Also, we recommend using Jobs Compute for job workloads.
