Hubert-Dudek
Databricks MVP

Hi, I tried again on another cluster but still working ok. Maybe try to change the runtime version. Additionally, you can check in Spark UI -> Environment -> Spark properties. Perhaps you have there some options set that change that behavior. I tried to find now in the options something related but couldn't.


My blog: https://databrickster.medium.com/