DLT job failed to parse timestamp string with T and Z.

Thomas_Zhang — Fri, 02 May 2025 12:26:10 GMT

Hi I am struggling with converting a timestamp string with T and Z to a timestamp column in my DLT job. here is the relevent code snippet:

trans_rules={'timestamp_value', '''to_timestamp(timstamp_str, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")'''}

In my DLT function, I have the following:

for col_name, expr_str in transformation_rules.items():

df = df.withColumn(col_name, F.expr(expr_str))

The DLT execution succeeded but the target column (timestamp_value) is null.

However, when I execute the similar command in a notebook, it works fine.

I am using serverless compute for DLT

Re: DLT job failed to parse timestamp string with T and Z.

Ayushi_Suthar — Fri, 02 May 2025 15:56:27 GMT

Hi @Thomas_Zhang , Good Day!

Can you give it a try with the below code format and check if it helps you to set the timezone with Z :

=======================

date_format( to_timestamp(`createdon`, 'yyyy-MM-dd\'T\'HH:mm:ss.SSSSSSSX'), 'yyyy-MM-dd\'T\'HH:mm:ss.SSSSSSSX' )

=======================

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi