Having a near identical issue just materializing a dataframe with `.toPandas()` an operation that now (14.3) takes 5 minutes used to take ~30s before on 10.4.
We are also related issues, see https://community.databricks.com/t5/get-started-discussions/databricks-connect-13-1-0-limitations/td-p/37096. However, this issue also highlights that the heartbeat that let you know a job was running on Databricks Con...
Seemingly our issue doesn't contain a "larger than max" error. Out of curiosity I tried to double the maxRecordsPerBatch to 20000 and reduced it to 200 and it didn't appear to help.```Fatal Python error: Segmentation faultThread 0x0000ffff7bfff120 (m...
We are having what appears to be the same segfault issue when running some of our larger chained functions (likely with quite large plans). However, we can reliably trigger this by looping over `withColumns` to increase the plan size. Our case is als...
