Datetime conversion on streaming tables

singh_tushar_14
Databricks Partner

I am using streaming tables to read from multiple parquet files and create a table in my raw layer.

Tech Stack -- dbt-databricks

While loading I have a column called "ets" in my source which is nothing "milliseconds since Jan 1st 1970." 

I want this to be converted into a timestamp format for which I use 

timestampadd(millisecond, ets, '1970-01-01')

ets = 1746046131518
expected_output = 2025-04-30T20:48:51.518+00:00
actual_output = 1969-12-08T18:54:02.046+00:00
  • Running it as a dry code on databricks notebook gives the correct output but when compiled through dbt and ran as a model same code gives a different output.

DBT Model code

SELECT
    *,
    {{ dbt.dateadd("MILLISECOND", "ets", "'1970-01-01'") }} AS ets_converted,
    {{ add_metadata_fields() }}
FROM STREAM read_files(
    'path/to/volume',
    format => 'parquet',
    header => TRUE
)

Compiled DBT Code from Databricks.

CREATE STREAMING TABLE `catalog_name`.`schema_name`.`table_name` (<inferredSchema>)
SELECT
  *,
  timestampadd(MILLISECOND, ets, '1970-01-01') AS ets_converted,
  CURRENT_TIMESTAMP() AS meta_loaded_at
FROM
  STREAM read_files(
    'path/to/volumes',
    format => 'parquet',
    header => TRUE
  )

I am not able to make an assumption is this thing wrong with dbt or streaming table or code.