โ07-27-2021 06:49 PM
Hi
I am getting "java.lang.NoClassDefFoundError: scala/Product$class" error while using Deequ 1.0.5 version. Please suggest fix to this problem or any work around
Error
Py4JJavaError Traceback (most recent call last) <command-2625366351750561> in <module>----> 1suggestionResult = ConstraintSuggestionRunner(spark)\ 2.onData(df)\ 3.addConstraintRule(DEFAULT())\ 4.run() 5 print(json.dumps(suggestionResult, indent=2))/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in addConstraintRule(self, constraintRule) 64for rule in constraintRule_jvm: 65 rule._set_jvm(self._jvm)---> 66rule_jvm = rule.rule_jvm 67 self._ConstraintSuggestionRunBuilder.addConstraintRule(rule_jvm) 68/databricks/python/lib/python3.8/site-packages/pydeequ/suggestions.py in rule_jvm(self) 184@property 185def rule_jvm(self):--> 186return self._deequSuggestions.rules.CategoricalRangeRule()
Code:
suggestionResult = ConstraintSuggestionRunner(spark) \ .onData(df) \ .addConstraintRule(DEFAULT()) \ .run() print(json.dumps(suggestionResult, indent=2))
Thanks
Ramabadran
โ09-02-2021 06:21 AM
Hi @ Ramabadran! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.
โ01-07-2022 06:20 AM
I am having a similar issue and receiving this error. It appears that from other forums it is related to scala version. @Ramabadranโ were you ever able to figure it out? @Kaniz Fatmaโ any response? Thank you.
โ01-07-2022 06:46 AM
Hi @David Foranโ and @Ramabadranโ , Which version are you using?
โ01-07-2022 06:49 AM
Youโre running this version that isnโt compatible with databricks runtime version. Deequ 1.0.5 had a problem with Spark 3, so itโs better to use version 1.2.2 that is the latest.
โ11-29-2022 10:34 PM
Hi @Ramabadranโ ,
You need to go Cluster setting -> Libraries -> Install Now ->
Choose MAVEN ->com.databricks:spark-xml_2.12:0.15.0 Enter this text.
Boom Issue resolved!!!
โ12-01-2022 03:27 AM
its seems like maven issue
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group