https://community.databricks.com/t5/data-engineering/spark-3-4-and-databricks-13-introduce-two-new-types-of/m-p/4786#M1404 <P><B>spark</B> 3.4 and <B>databricks</B> 13 introduce two new types of timestamps for handling time zone information:</P><P>- TIMESTAMP WITH LOCAL TIME ZONE: This type assumes that the input data is in the session's local time zone and converts it to UTC before processing. This allows for consistent results across different time zones and daylight saving changes.</P><P>- TIMESTAMP WITHOUT TIME ZONE: This type treats the input data as time zone insensitive and performs no conversion. This is useful for data that does not depend on the time zones context, such as event logs or sensor readings.</P><P></P><P><span class="lia-inline-image-display-wrapper" image-alt="timezone"><img src="https://community.databricks.com/t5/image/serverpage/image-id/238iDE3DE4B9D94371FB/image-size/large?v=v2&amp;px=999" role="button" title="timezone" alt="timezone" /></span>&nbsp;</P><P></P> Fri, 05 May 2023 08:58:28 GMT Hubert-Dudek 2023-05-05T08:58:28Z https://community.databricks.com/t5/data-engineering/spark-3-4-and-databricks-13-introduce-two-new-types-of/m-p/4786#M1404 <P><B>spark</B> 3.4 and <B>databricks</B> 13 introduce two new types of timestamps for handling time zone information:</P><P>- TIMESTAMP WITH LOCAL TIME ZONE: This type assumes that the input data is in the session's local time zone and converts it to UTC before processing. This allows for consistent results across different time zones and daylight saving changes.</P><P>- TIMESTAMP WITHOUT TIME ZONE: This type treats the input data as time zone insensitive and performs no conversion. This is useful for data that does not depend on the time zones context, such as event logs or sensor readings.</P><P></P><P><span class="lia-inline-image-display-wrapper" image-alt="timezone"><img src="https://community.databricks.com/t5/image/serverpage/image-id/238iDE3DE4B9D94371FB/image-size/large?v=v2&amp;px=999" role="button" title="timezone" alt="timezone" /></span>&nbsp;</P><P></P> Fri, 05 May 2023 08:58:28 GMT https://community.databricks.com/t5/data-engineering/spark-3-4-and-databricks-13-introduce-two-new-types-of/m-p/4786#M1404 Hubert-Dudek 2023-05-05T08:58:28Z https://community.databricks.com/t5/data-engineering/spark-3-4-and-databricks-13-introduce-two-new-types-of/m-p/4787#M1405 <P>This is helpful! Timestamps are always the reason to mess up the business logic as we know.</P> Mon, 08 May 2023 06:27:41 GMT https://community.databricks.com/t5/data-engineering/spark-3-4-and-databricks-13-introduce-two-new-types-of/m-p/4787#M1405 Anonymous 2023-05-08T06:27:41Z