Hello @George Petrou​ 

Could you please try setting the below Spark configuration on the cluster?

spark.driver.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true

spark.executor.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true