Malthe
Valued Contributor II

We're not using any external libraries; this is just vanilla PySpark running on the latest serverless runtime (environment version 5). The segmentation fault must come from some of the Databricks software that powers the serverless platform, perhaps telemetry.