topic Re: java.lang.NoClassDefFoundError: scala/Product$class in Data Engineering

java.lang.NoClassDefFoundError: scala/Product$class

Ramabadran — Wed, 28 Jul 2021 01:49:34 GMT

I am getting "java.lang.NoClassDefFoundError: scala/Product$class" error while using Deequ 1.0.5 version. Please suggest fix to this problem or any work around

Error

Py4JJavaError Traceback (most recent call last) <command-2625366351750561> in <module>----> 1suggestionResult = ConstraintSuggestionRunner(spark)\ 2.onData(df)\ 3.addConstraintRule(DEFAULT())\ 4.run() 5 print(json.dumps(suggestionResult, indent=2))/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in addConstraintRule(self, constraintRule) 64for rule in constraintRule_jvm: 65 rule._set_jvm(self._jvm)---> 66rule_jvm = rule.rule_jvm 67 self._ConstraintSuggestionRunBuilder.addConstraintRule(rule_jvm) 68/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in rule_jvm(self) 184@property 185def rule_jvm(self):--> 186return self._deequSuggestions.rules.CategoricalRangeRule()

Code:

suggestionResult = ConstraintSuggestionRunner(spark) \ .onData(df) \ .addConstraintRule(DEFAULT()) \ .run() print(json.dumps(suggestionResult, indent=2))

Thanks

Ramabadran

Re: java.lang.NoClassDefFoundError: scala/Product$class

DaveForan — Fri, 07 Jan 2022 14:20:39 GMT

I am having a similar issue and receiving this error. It appears that from other forums it is related to scala version. @Ramabadran were you ever able to figure it out? @Kaniz Fatma any response? Thank you.

Re: java.lang.NoClassDefFoundError: scala/Product$class

Saiki — Wed, 30 Nov 2022 06:34:47 GMT

Hi @Ramabadran ,

You need to go Cluster setting -> Libraries -> Install Now ->

Choose MAVEN ->com.databricks:spark-xml_2.12:0.15.0 Enter this text.

Boom Issue resolved!!!

Re: java.lang.NoClassDefFoundError: scala/Product$class

mcwir — Thu, 01 Dec 2022 11:27:29 GMT

its seems like maven issue