cancel
Showing results for 
Search instead for 
Did you mean: 
Technical Blog
Explore in-depth articles, tutorials, and insights on data analytics and machine learning in the Databricks Technical Blog. Stay updated on industry trends, best practices, and advanced techniques.
cancel
Showing results for 
Search instead for 
Did you mean: 
ajiang22
Databricks Employee
Databricks Employee

Enterprises store timestamps to understand when events occur, establish their sequence, and measure the time between them. For most analytical workloads, microsecond precision is sufficient. However, in fields like finance, manufacturing, and scientific research, where high-frequency data can contain multiple events within the same microsecond, nanosecond precision is critical.

Nanosecond timestamps, now in Beta, brings nanosecond-precision timestamps to enterprises on Databricks. With TIMESTAMP(9) and TIMESTAMP_NTZ(9), you can retain up to nine fractional-second digits in a native timestamp column and analyze high-frequency events without precision loss or workarounds that store timestamps as strings or integers. Timestamp nano support is fully open-sourced and integrated within Apache Spark™, Delta Lake, and Apache Iceberg.

timestamp-nano-only.png

What you can do with Timestamp Nano

The applications of Timestamp Nano extend to numerous domains. Enterprises can:

  • Analyze financial markets by preserving the order of trades, quotes, and order-book updates for backtesting, execution analysis, and audit trails.
  • Investigate industrial systems by ordering tightly spaced readings from high-speed sensors and equipment.
  • Conduct scientific experiments by capturing measurements from high-speed instruments.

Timestamp Nano reduces friction throughout the lifecycle of high-frequency data, from ingestion and preparation to analysis. To demonstrate, consider a question from the head of quantitative research at a trading firm:

Did our strategy react to the quote that was actually available when each trade occurred?

The data needed to answer this question is a stream of market events. Quotes and trades arrive in a clear order, but several can occur inside one microsecond:

09:30:00.123456701  QUOTE
09:30:00.123456742  TRADE
09:30:00.123456799  QUOTE

At microsecond precision, every event becomes 09:30:00.123456. The rows remain, but the sequence does not. Here is how Timestamp Nano preserves the information needed to answer the research question.

  • Create a native nanosecond timestamp column. The team defines event_time as TIMESTAMP_NTZ(9), keeping the event time in the table schema rather than hiding it inside a string or integer.
CREATE TABLE market_events (
  symbol STRING,
  event_type STRING,
  event_time TIMESTAMP_NTZ(9),
  price DECIMAL(18, 4)
)
  • Reconstruct the original sequence. Once the table retains all nine digits, basic SQL can sort events in the order in which the source recorded them.
SELECT symbol, event_type, event_time, price
FROM market_events
ORDER BY event_time;
  • Answer the research question. The team can now see which quote came before each trade and which came after it.

The same pattern applies outside financial markets. A manufacturing team can preserve the order of sensor readings around a fault. A research team can reconstruct measurements captured during a fast-moving experiment. In each case, the nanosecond precision remains part of the data from source to analysis.

Get started with Timestamp Nano

Timestamp Nano is available now in Beta. Get started today by reading the TIMESTAMP documentation and TIMESTAMP_NTZ documentation, and testing Timestamp Nano with your market, sensor, or scientific data.

If you have questions or feedback, please reach out to your Databricks account team.