Change datetime format from one to another without changing datatype in databricks sql
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-26-2024 01:10 AM
Change datetime"2002-01-01T00:00:00.000" to 'MM/dd/yyyy HH:mm:ss' format without changing datatype/ having it in datetime data type
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-26-2024 05:31 AM
I presume then that the "datetime" string you provided is a string column?
If so, try the following code. This will not perform any transformation to timestamps:
from pyspark.sql.functions import lit, split, concat
data = [{"date_string": "2002-01-01T00:00:00.000"}]
data_df = spark.createDataFrame(data) \
.withColumn(
"date_string_reformatted", concat(
lit(split("date_string", "-")[1]),
lit("-"),
lit(split(split("date_string", "-")[2], "T")[0]),
lit("-"),
lit(split("date_string", "-")[0]),
lit(" "),
lit(split(split(split("date_string", "-")[2], "T")[1], "\.")[0]),
)
)
data_df.display() The result:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-26-2024 06:03 AM
Hi @Mister-Dinky ,
I think what @richakamat130 would like to obtain is to format datetime according to his needs, but to keep orginal data format (in this case datetime). Unfortunately, this is not possible in spark. If you want to have different display format then you need to have this conversion.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-26-2024 06:10 AM
Ah, the "without changing datatype/ having it in datetime data type" can be interpreted in that way as well. Then I agree with your statement.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-26-2024 06:53 AM
Hi @Mister-Dinky ,
As @szymon_dybczak if you have a datetime, then you have a datetime.
What you see is just a format of defined in the Databricks UI. Other applications may display it differently depending on the defaults, regional formats etc.
If you want to see datetime displayed differently in Databricks UI, then just navigate to your user settings and change it accordingly to your preferences: