https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25751#M17958 <UL><LI>Make sure scala version is 2.12. Also if the library is based on Spark 2.x then update that library to Spark 2.4.5 for Scala 2.12 support.</LI><LI>For SBT projects use crossScalaVersion setting to build libraries for both versions of Scala</LI></UL> Mon, 07 Jun 2021 22:57:52 GMT User16789201666 2021-06-07T22:57:52Z https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25750#M17957 <P>What are some guidelines for migrating to DBR 7/Spark 3?</P> Mon, 07 Jun 2021 22:57:12 GMT https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25750#M17957 User16789201666 2021-06-07T22:57:12Z https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25751#M17958 <UL><LI>Make sure scala version is 2.12. Also if the library is based on Spark 2.x then update that library to Spark 2.4.5 for Scala 2.12 support.</LI><LI>For SBT projects use crossScalaVersion setting to build libraries for both versions of Scala</LI></UL> Mon, 07 Jun 2021 22:57:52 GMT https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25751#M17958 User16789201666 2021-06-07T22:57:52Z https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25752#M17959 <P>Please refer to the below reference for switching to DBR 7.x</P><P></P><P>We have extended our DBR 6.4 support until December 2021, <B>DBR 6.4 extended support </B>- Release notes: <A href="https://docs.databricks.com/release-notes/runtime/6.4x.html" target="test_blank">https://docs.databricks.com/release-notes/runtime/6.4x.html</A></P><P></P><P><B>Migration guide to DBR 7.x: </B></P><P><A href="https://docs.databricks.com/release-notes/runtime/7.x-migration.html" target="test_blank">https://docs.databricks.com/release-notes/runtime/7.x-migration.html</A> <A href="https://spark.apache.org/docs/latest/sql-migration-guide.html" target="test_blank">https://spark.apache.org/docs/latest/sql-migration-guide.html</A></P><P></P><P><B>New Features and Changes in Spark 3 (DBR 7.x)</B></P><P>DBR 7.x AQE: <A href="https://databricks.com/blog/2020/05/29/adaptive-query-execution-speeding-up-spark-sql-at-runtime.html" target="test_blank">https://databricks.com/blog/2020/05/29/adaptive-query-execution-speeding-up-spark-sql-at-runtime.html</A></P><P>New changes to Dates and Timestamp: <A href="https://databricks.com/blog/2020/07/22/a-comprehensive-look-at-dates-and-timestamps-in-apache-spark-3-0.html" target="test_blank">https://databricks.com/blog/2020/07/22/a-comprehensive-look-at-dates-and-timestamps-in-apache-spark-3-0.html</A></P><P>New changes to Pandas UDFs and Python Type hints: <A href="https://databricks.com/blog/2020/05/20/new-pandas-udfs-and-python-type-hints-in-the-upcoming-release-of-apache-spark-3-0.html" target="test_blank">https://databricks.com/blog/2020/05/20/new-pandas-udfs-and-python-type-hints-in-the-upcoming-release-of-apache-spark-3-0.html</A></P><P></P><P>Get an overview on DBR 7.3 LTS: <A href="https://databricks.com/blog/2021/03/08/upgrade-production-workloads-to-be-safer-easier-and-faster-with-databricks-runtime-7-3-lts.html?_ga=2.6592177.241838665.1633224496-85840594.1621756477" alt="https://databricks.com/blog/2021/03/08/upgrade-production-workloads-to-be-safer-easier-and-faster-with-databricks-runtime-7-3-lts.html?_ga=2.6592177.241838665.1633224496-85840594.1621756477" target="_blank">https://databricks.com/blog/2021/03/08/upgrade-production-workloads-to-be-safer-easier-and-faster-with-databricks-runtime-7-3-lts.html</A></P> Sun, 03 Oct 2021 02:11:34 GMT https://community.databricks.com/t5/data-engineering/what-are-some-guidelines-for-migrating-to-dbr-7-spark-3/m-p/25752#M17959 shan_chandra 2021-10-03T02:11:34Z