dbruehlmeier
Contributor

Hi Robin

Do you use Databricks Connect creating spark Session?

from databricks.connect import DatabricksSession
spark = DatabricksSession.builder.getOrCreate()
We are facing same issue on single user access cluster and in jobs.