How can I add a duration in milliseconds to a timestamp?

Merchiv
New Contributor III

Let's say I have a DataFrame with a timestamp and an offset column in milliseconds respectively in the timestamp and long format.

E.g.

from datetime import datetime
df = spark.createDataFrame(
    [
        (datetime(2021, 1, 1), 1500, ),
        (datetime(2021, 1, 2), 1200, )
    ],
    ["timestamp", "offsetmillis", ],
)

Now I want to add these offsets to the datetime, so that I get:

2021-01-01T00:00:01.500 and 2021-01-0T00:00:01.200

If I add these directly I get an error about type mismatch, which does make sense:

[DATATYPE_MISMATCH.BINARY_OP_DIFF_TYPES] Cannot resolve "(timestamp + offsetmillis)" due to data type mismatch: the left and right operands of the binary operator have incompatible types ("TIMESTAMP" and "BIGINT")

However I'm not sure how I can best cast this to a duration or interval.