[INCONSISTENT_BEHAVIOR_CROSS_VERSION.PARSE_DATETIME_BY_NEW_PARSER]

[117074]

Hi all,

I'm trying to join 2 views in SQL editor for some analysis. I get the following error:

[INCONSISTENT_BEHAVIOR_CROSS_VERSION.PARSE_DATETIME_BY_NEW_PARSER] You may get a different result due to the upgrading to Spark >= 3.0: Fail to parse '22/12/978' in the new parser. You can set "legacy_time_parser_policy" to "LEGACY" to restore the behavior before Spark 3.0, or set to "CORRECTED" and treat it as an invalid datetime string.

I've tried correcting this by using the command: 'SET legacy_time_parser_policy = legacy;' before my code but it still errors. 

Is anyone able to advise on how to solve this? 

Thanks

[117074]

Hi Kaniz, I found the equivalent SQL code for this but it didn't seem to store the operation past the execution. I.e I would run the code to configure settings, then run the troublesome code afterwards and still get the same result. 

The problem has been identified as source data missing a digit for date of birth year, and therefore having results before 1000AD. This has been rectified at source. 

Thank you for your time in answering!