topic Re: java.lang.NoSuchMethodError in databricks in Data Engineering

java.lang.NoSuchMethodError in databricks

sarvesh242 — Tue, 12 Apr 2022 08:07:09 GMT

I have created a package in scala. Now, I am calling a method from that package and using it in my notebook. During run time, it throws me an error java.lang.NoSuchMethodError. The method exists in the package but still, I am getting this error. Please guide me regarding the same.

Re: java.lang.NoSuchMethodError in databricks

sarvesh242 — Wed, 13 Apr 2022 08:48:03 GMT

scala version 2.11 with spark 2.4.3

Re: java.lang.NoSuchMethodError in databricks

sarvesh242 — Wed, 13 Apr 2022 11:51:37 GMT

Hi! @Kaniz Fatma . I am using dependencies for scala 2.11. Can you please elaborate on the above point

"mixing different scala versions".

Thanks

Re: java.lang.NoSuchMethodError in databricks

sarvesh242 — Thu, 14 Apr 2022 13:45:48 GMT

Hi! @Kaniz Fatma . I am using scala version 2.11 with spark 2.4.3.

According to Apache spark official website 😞https://spark.apache.org/docs/2.4.3/#:~:text=For%20the%20Scala%20API%2C%20Spark,x.)

Spark 2.4.3 uses Scala 2.12.

(https://spark.apache.org/docs/2.4.0/#:~:text=For%20the%20Scala%20API%2C%20Spark,0%20uses%20Scala%202.11.)

Spark 2.4. 0 uses Scala 2.11.

I have two options with respect to Spark/scala version. Does any option has edge over the other ?

Thanks!