- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 10:21 PM
I am thinking of migrating the spark 2.4 to 3.0, what should I know to take care of changes thet I need to look at while migrating
- Labels:
-
Spark
-
Spark Code
-
Spark Migration
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 10:25 PM
I see there are many changes that need to take care of if you have used coding in spark 2.4
the changes are in Data set API Statement
Builtin UDF and functions
More you can get from spark Documentation
https://spark.apache.org/docs/latest/sql-migration-guide.html#upgrading-from-spark-sql-24-to-30
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 10:25 PM
I see there are many changes that need to take care of if you have used coding in spark 2.4
the changes are in Data set API Statement
Builtin UDF and functions
More you can get from spark Documentation
https://spark.apache.org/docs/latest/sql-migration-guide.html#upgrading-from-spark-sql-24-to-30