How to add a new datetime column to a spark dataFrame from existing timestamp column

[User16790091296]

I have a data frame in Spark that has a column timestamp. I want to add a new column to this data frame that has the DateTime in the below format created from this existing timestamp column.

“YYYY-MM-DD HH:MM:SS”

[Srikanth_Gupta_]

val df = Seq(("2021-11-05 02:46:47.154410"),("2019-10-05 2:46:47.154410")).toDF("old_column")

display(df)

import org.apache.spark.sql.functions._

val df2 = df.withColumn("new_column", from_unixtime(unix_timestamp(col("old_column"), "yyyy-MM-dd HH:mm:ss.SSSSSS"),"yyyy-MM-dd HH:mm:ss"))

display(df2)

I have tested this and this should work