Summary:
from_utc_timestamp(current_timestamp(), '<tz>') produces an incorrect (future-shifted) timestamp whenever the Spark session timezone is already set to something other than UTC. This is a very common pattern for teams stamping dp_load_ts/created_at/audit columns, and the current docs don't warn against it.
Root cause:
current_timestamp() returns the current time in the session's configured timezone (spark.sql.session.timeZone / cluster timezone setting), not UTC. from_utc_timestamp(ts, tz) always treats its input ts as if it were UTC, regardless of what it actually is, and converts it into tz.
So if a cluster's session timezone is set to Asia/Bangkok (UTC+7):
SELECT current_timezone();
-- Asia/Bangkok
SELECT
current_timestamp() AS raw_ts, -- correct Bangkok wall-clock time
from_utc_timestamp(current_timestamp(), 'Asia/Bangkok') AS double_shifted -- Bangkok time + 7 more hours
double_shifted ends up 7 hours ahead of the actual current time, because current_timestamp() is already local, and from_utc_timestamp adds the offset a second time on top of it.
Why this matters:
from_utc_timestamp(current_timestamp(), '<local tz>') is a very natural-looking pattern to write when a team wants a "load timestamp in our local timezone," and it's silently wrong whenever the session timezone isn't UTC. It produces no error — just a plausible-looking but incorrect future timestamp — so it can go unnoticed in production pipelines for a long time.
Suggested fix:
Add an explicit warning/example on the from_utc_timestamp (and ideally current_timestamp()) doc pages, something like:
⚠️ from_utc_timestamp always assumes its input is UTC. If your session timezone (current_timezone()) is not UTC, do not pass current_timestamp() directly into from_utc_timestamp() — this will double-apply the timezone offset. If the session timezone already matches your target timezone, use current_timestamp() directly instead.
