one of the date datatype format issue in pysaprk

[KVNARK]

if anyone has encountered this date type format - 6/15/25 12:00 AM could you mention the right formatting to be used in Pyspark.

Thanks in advance!

[UmaMahesh1]

Hi @KVNARK .​ 

You can use the below formatting to get it into a timestamp format...

to_timestamp('6/15/25 12:00 AM', 'MM/dd/yy hh:mm aa')

This will give you the timestamp like this -- 2025-06-15T00:00:00.000+0000

If you want only the date part, you can use to_date function.

Hope this helps...Cheers..

[UmaMahesh1]

FYI... You also have to set set spark.sql.legacy.timeParserPolicy = LEGACY for the above to work..

[Hubert-Dudek]

Without legacy, it will also work.

SELECT to_timestamp('6/15/23 12:00 AM', 'M/dd/yy h:mm a')

[Kaniz]

Hi @KVNARK .​, We haven’t heard from you since the last response from @Uma Maheswara Rao Desula​ and @Hubert Dudek​ I was checking back to see if his suggestions helped you.

Or else, If you have any solution, please do share that with the community as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.