It would help a lot if you attach a notebook or copy the full code that demonstrates the problem. This would make it easy for someone to copy/paste, run, and troubleshoot it and you are more likely to get effective help that way 😉

Also, since your Databricks notebook runs on some kind of Databricks compute, it might be useful to mention what are the parameters of that compute. If it is not serverless, you could include a screenshot of the cluster configuration. Things that could matter in troubleshooting are the access mode and whether it's a single user cluster and what version of Databricks does it run.

I hope this helps!