Anonymous
Not applicable

@Dinu Sukumara​ : My take on your question -

If you want to preserve the timestamp values exactly as they are in the source database, without any timezone conversion, you can follow these steps:

  1. Set the Cluster Timezone: Keep the cluster timezone as UTC, as you mentioned.
  2. Adjust Session Timezone: Before reading the data from the source database, you can adjust the session timezone in Databricks to match the timezone of the source data.
spark.conf.set("spark.sql.session.timeZone", "<source_timezone>")

3 . Read Data from Source: Use the JDBC read functionality in Databricks to extract the data from the source database.

df = spark.read.format("jdbc").option("url", "<jdbc_url>").option("dbtable", "<table_name>").load()