We are running a job on a cluster with DBR 10.4 LTS Spark 3.2.1 and scala 2.12. Cluster is using 4 workers (spot instances) .The driver is not a spot instance
Randomly ( only on one environment and not all executions), we are getting the following error : java.lang.NoClassDefFoundError: Could not initialize class path.to.one.of.our.Class$
The class does exists and is present in the jar we are loading as a dependant library in the job task definition. The jar was compiled with the same versions (3.2.1/2.12)
The environment is identical to other working ones and as I 've mentionned, just relaunching the same failing job once makes it succeed, without changing anything.
Any ideas regarding what could the problem possibly be ?
