one of the date datatype format issue in pysaprk

[KVNARK]

if anyone has encountered this date type format - 6/15/25 12:00 AM could you mention the right formatting to be used in Pyspark.

Thanks in advance!

[UmaMahesh1]

Hi @KVNARK .​ 

You can use the below formatting to get it into a timestamp format...

to_timestamp('6/15/25 12:00 AM', 'MM/dd/yy hh:mm aa')

This will give you the timestamp like this -- 2025-06-15T00:00:00.000+0000

If you want only the date part, you can use to_date function.

Hope this helps...Cheers..

[UmaMahesh1]

FYI... You also have to set set spark.sql.legacy.timeParserPolicy = LEGACY for the above to work..

[Hubert-Dudek]

Without legacy, it will also work.

SELECT to_timestamp('6/15/23 12:00 AM', 'M/dd/yy h:mm a')