Could not initialize class error

Mohit_m
Databricks Employee
Databricks Employee

‎06-27-2022 05:24 AM

User is running a job triggered from ADF in Databricks. In this job they need to use custom libraries that are in jars. Most of the times jobs are running fine, however sometimes it fails with:

java.lang.NoClassDefFoundError: Could not initialize

Any suggestions?

0 Kudos

Mohit_m
Databricks Employee
Databricks Employee

‎06-27-2022 05:25 AM

Can you please check if there are more than one jar containing this class . If multiple jars of the same type are available on the cluster, then there is no guarantee of JVM picking the proper classes for processing, which results in the intermittent behaviour of job run.

View solution in original post

BillSchneider
Databricks Partner
In response to Mohit_m

‎04-23-2024 07:37 AM

I have a similar issue.

There is an intermittent issue with NoClassDefFoundError which looks like it's coming after several layers of ObjectInputStream.readObject -- perhaps trying to deserialize a reference in a shuffled object

The class appears once in a JAR, which is deployed once as a Library JAR for the job.

Most of the time it works fine, holding everything else constant, but there are occasional failures with this error. What could cause this failure to be intermittent and not consistent?