Debayan
Databricks Employee
Databricks Employee

Hi, this looks like an issue with compatibility, could you please check if the dependency packages are also installed?

Also, I saw a note in the end of the link above, could you please confirm if this was done?

Note: You need to install the Sedona libraries via init script because the libraries installed via UI are installed after the cluster has already started, and therefore the classes specified by the config 

spark.sql.extensions

spark.serializer

, and 

spark.kryo.registrator

 are not available at startup time.