Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-12-2023 02:50 AM
@Pien Derkx You can set the spark sql formatter to LEGACY and try your steps. Below is what worked for me.
spark.conf.set("spark.sql.legacy.timeParserPolicy", "LEGACY")
df.withColumn("date_expected", to_date(col("dateinput", "uwwyyyy")).show()dateinput -> 3312023
Output came for dateexected -> 2023-03-29
Please try and let me know if this works.