Hi, I ran into the same error starting late last week.

The issue was caused by dependency version conflicts between my custom package and the libraries preinstalled on the Databricks serverless environment.

I fixed it by pinning all my package dependencies to the versions already provided by the serverless cluster, which resolved the error.