- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-06-2025 04:09 AM
The error you are encountering seems to be related to the runtime includes Apache Spark 3.5.x, which defines the constructor as public SparkListenerApplicationEnd(long time), while a conflicting library (----ws_3_5--core--core-hive-2.3__hadoop-3.2_2.12_deploy.jar) expects a different version of the constructor: public SparkListenerApplicationEnd(long time, scala.Option<Object> exitCode).
This issue occurs because the JVM attempts to use the single-parameter constructor but fails due to the conflicting library expecting the two-parameter version, leading to the NoSuchMethodError.
To resolve this issue, you can try the following steps:
-
Identify and Remove Conflicting Libraries: Check your dependencies and remove or update the conflicting library that includes the SparkListenerApplicationEnd class with the two-parameter constructor. Ensure that all libraries are compatible with Apache Spark 3.5.x.
-
Update Dependencies: Ensure that all your project dependencies are updated to versions compatible with Databricks runtime 15.5 LTS and Apache Spark 3.5.x.