cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

java.lang.NoClassDefFoundError: scala/Product$class

Ramabadran
New Contributor II

Hi

I am getting "java.lang.NoClassDefFoundError: scala/Product$class" error while using Deequ 1.0.5 version. Please suggest fix to this problem or any work around

Error

Py4JJavaError Traceback (most recent call last) <command-2625366351750561> in <module>----> 1suggestionResult = ConstraintSuggestionRunner(spark)\ 2.onData(df)\ 3.addConstraintRule(DEFAULT())\ 4.run() 5 print(json.dumps(suggestionResult, indent=2))/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in addConstraintRule(self, constraintRule) 64for rule in constraintRule_jvm: 65 rule._set_jvm(self._jvm)---> 66rule_jvm = rule.rule_jvm 67 self._ConstraintSuggestionRunBuilder.addConstraintRule(rule_jvm) 68/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in rule_jvm(self) 184@property 185def rule_jvm(self):--> 186return self._deequSuggestions.rules.CategoricalRangeRule()

Code:

suggestionResult = ConstraintSuggestionRunner(spark) \ .onData(df) \ .addConstraintRule(DEFAULT()) \ .run() print(json.dumps(suggestionResult, indent=2))

Thanks

Ramabadran

6 REPLIES 6

Kaniz
Community Manager
Community Manager

Hi @ Ramabadran! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.

DaveForan
New Contributor II

I am having a similar issue and receiving this error. It appears that from other forums it is related to scala version. @Ramabadran​ were you ever able to figure it out? @Kaniz Fatma​ any response? Thank you.

Kaniz
Community Manager
Community Manager

Hi @David Foran​ and @Ramabadran​ , Which version are you using?

Kaniz
Community Manager
Community Manager

You’re running this version that isn’t compatible with databricks runtime version. Deequ 1.0.5 had a problem with Spark 3, so it’s better to use version 1.2.2 that is the latest.

Saiki
New Contributor II

Hi @Ramabadran​ ,

You need to go Cluster setting -> Libraries -> Install Now ->

Choose MAVEN ->com.databricks:spark-xml_2.12:0.15.0 Enter this text.

Boom Issue resolved!!!

mcwir
Contributor

its seems like maven issue

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.