shan_chandra
Databricks Employee
Databricks Employee

ProtoSerializer is known to be prone to StackOverflow when very deep query plans are being used. Could you please try increasing the stack size of the client JVM with the spark conf -spark.driver.extraJavaOptions which can be defined in ${spark_home}/conf/spark-defaults.conf  (e.g,

spark.driver.extraJavaOptions -Xss4M

), where ${spark_home}  can be found with databricks-connect get-spark-home

View solution in original post