topic Re: NoClassDefFoundError: scala/Product$class in Data Engineering

NoClassDefFoundError: scala/Product$class

YSDPrasad — Fri, 10 Mar 2023 19:45:22 GMT

import com.microsoft.azure.sqldb.spark.config.Config

import com.microsoft.azure.sqldb.spark.connect._

import com.microsoft.azure.sqldb.spark.query._

val query = "Truncate table tablename"

val config = Config(Map(

"url" -> dbutils.secrets.get(scope = "azurekeyvault-scope", key = "DW-URL"),

"databaseName" -> dbutils.secrets.get(scope = "azurekeyvault-scope", key = "DW-DBName"),

"user" -> dbutils.secrets.get(scope = "azurekeyvault-scope", key = "DW-Username"),

"password" -> dbutils.secrets.get(scope = "azurekeyvault-scope", key = "DW-Password"),

"queryCustom" -> query

))

sqlContext.sqlDBQuery(config)

While executing the above command in cluster 12.0 runtime version i am facing an error like NoClassDefFoundError

Re: NoClassDefFoundError: scala/Product$class

Anonymous — Mon, 13 Mar 2023 07:22:33 GMT

@Someswara Durga Prasad Yaralgadda :

The NoClassDefFoundError error occurs when a class that was available during the compile time is not available at the runtime. This could be due to a few reasons, including a missing dependency or an incompatible version of a dependency.

In your case, it seems like the error is related to the Azure SQL DB Spark connector library. The missing class could be a part of this library. You can try the following steps to resolve the issue:

Check that the necessary Azure SQL DB Spark connector library is available in your cluster's runtime environment. You can try to add the dependency explicitly using the %AddJar magic command.
Make sure that the version of the Azure SQL DB Spark connector library matches the version that is compatible with your Spark cluster's runtime version.

Re: NoClassDefFoundError: scala/Product$class

YSDPrasad — Tue, 21 Mar 2023 10:10:41 GMT

Yes suteja by adding additional jar files I can resolve this issue but while running the read and write operation by connecting sqldb there I am facing error. I found different code to execute truncate command now it is working fine.

Re: NoClassDefFoundError: scala/Product$class

Lraghav — Tue, 12 Sep 2023 15:15:19 GMT

Hi @YSDPrasad , could you please let me know which additonal jar files need to install to resolve this ?