• Make sure scala version is 2.12. Also if the library is based on Spark 2.x then update that library to Spark 2.4.5 for Scala 2.12 support.
• For SBT projects use crossScalaVersion setting to build libraries for both versions of Scala

Please refer to the below reference for switching to DBR 7.x

We have extended our DBR 6.4 support until December 2021, DBR 6.4 extended support - Release notes: https://docs.databricks.com/release-notes/runtime/6.4x.html

Migration guide to DBR 7.x:

https://docs.databricks.com/release-notes/runtime/7.x-migration.html https://spark.apache.org/docs/latest/sql-migration-guide.html

New Features and Changes in Spark 3 (DBR 7.x)

DBR 7.x AQE: https://databricks.com/blog/2020/05/29/adaptive-query-execution-speeding-up-spark-sql-at-runtime.htm...

New changes to Dates and Timestamp: https://databricks.com/blog/2020/07/22/a-comprehensive-look-at-dates-and-timestamps-in-apache-spark-...

New changes to Pandas UDFs and Python Type hints: https://databricks.com/blog/2020/05/20/new-pandas-udfs-and-python-type-hints-in-the-upcoming-release...

Get an overview on DBR 7.3 LTS: https://databricks.com/blog/2021/03/08/upgrade-production-workloads-to-be-safer-easier-and-faster-wi...