-werners-
Esteemed Contributor III

f.e. there is a spark option to enable the 'old' date handling.

You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0.

Frankly I am not a fan of that approach as Spark 3 gives you a lot of interesting date functions.

So what you could do is to first identify where you have date columns, and explicitly cast them to dates with the to_date function.

View solution in original post