We tried moving our scala script from standalone cluster to databricks platform. 

Our script is compatible with following version:

Spark: 2.4.8 Scala: 2.11.12

The databricks cluster has spark/scala following with version:

Spark: 3.2.1. Scala: 2.12

1: we are not able to run the script through Jar in "Yarn" mode

2: Getting issues while parsing dates

Few date functions were not working on the current version of spark so we had to change the timeParserPolicy in configuration when passed.

Corresponding Error: Fail to parse '2022-07-07 02:14:26.233' in the new parser. You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0, or set to CORRECTED and treat it as an invalid datetime string.

3: Getting issue in using ".toList" function.

var df2=df.select("clicked_at").distinct().map(f=>f.getDate(0)).collect().toList

(this toList function is causing error)

Corresponding Error: Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps

Any help on this will be really helpful