Hi
I'm using Azure databricks 10.4 LTS (includes Apache Spark 3.2.1, Scala 2.12). I'm trying to convert 2 columns from string data type to timestamp data type . My date columns are in below format
2/18/2021 7:20:12 PM
So I wrote following command
from pyspark.sql.functions import to_timestamp
(
spark.read
.table("convertToTimeStamp")
.withColumn("JoiningDate",to_timestamp("JoiningDate", "dd/MM/yyyy HH:mm:ss"))
.withColumn("EndDate",to_timestamp("EndDate", "dd/MM/yyyy HH:mm:ss"))
.write
.format("delta")
.mode("overwrite")
.option("overwriteSchema", True)
.saveAsTable("convertToTimeStamp")
)
But I'm getting error message saying that
raise Py4JJavaError(
"An error occurred while calling {0}{1}{2}.\n".
format(target_id, ".", name), value)
Can you please help me to resolve the issue?
