I am trying to initialize class org.apache.spark.scheduler.SparkListenerApplicationEnd with databricks 15.4LTS.

Spark 3.5.0 expects a single argument constructor for org.apache.spark.scheduler.SparkListenerApplicationEnd(long time)

Whereas the class packaged in Databricks jar "----ws_3_5--core--core-hive-2.3__hadoop-3.2_2.12_deploy.jar" expects a 2 argument constructor i.e.

org.apache.spark.scheduler.SparkListenerApplicationEnd(long time, scala.Option<Object> exitCode)

This 2 argument constructor is in line with Spark 4.0.0-preview2 version and NOT IN spark version 3.5.0

This is causing a conflict, can you please check this version issue in the Databricks cluster binaries.