I use databricks-connect, and spark jobs related spark dataframe works good. But, when I trigger spark ml code, I am getting errors.

For example, after executing in the code: https://docs.databricks.com/_static/notebooks/gbt-regression.html

pipelineModel = pipeline.fit(train)

22/11/04 09:28:15 ERROR Instrumentation: java.io.IOException: unexpected exception type
	at java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1750)
	at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1280)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196)
---------------------------
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
---------------------------
Caused by: java.lang.IllegalArgumentException: Illegal lambda deserialization
	at scala.runtime.LambdaDeserializer$.makeCallSite$1(LambdaDeserializer.scala:89)
	at scala.runtime.LambdaDeserializer$.deserializeLambda(LambdaDeserializer.scala:114)
	at scala.runtime.LambdaDeserialize.deserializeLambda(LambdaDeserialize.java:38)
---------------------------
py4j.protocol.Py4JJavaError: An error occurred while calling o806.fit.
: java.io.IOException: unexpected exception type
	at java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1750)
	at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1280)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196)
---------------------------
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
---------------------------
Caused by: java.lang.IllegalArgumentException: Illegal lambda deserialization
	at scala.runtime.LambdaDeserializer$.makeCallSite$1(LambdaDeserializer.scala:89)
	at scala.runtime.LambdaDeserializer$.deserializeLambda(LambdaDeserializer.scala:114)
	at scala.runtime.LambdaDeserialize.deserializeLambda(LambdaDeserialize.java:38)

Does anyone know how to fix it?