Enterprises store timestamps to understand when events occur, establish their sequence, and measure the time between them. For most analytical workloads, microsecond precision is sufficient. However, in fields like finance, manufacturing, and scientific research, where high-frequency data can contain multiple events within the same microsecond, nanosecond precision is critical.
Nanosecond timestamps, now in Beta, brings nanosecond-precision timestamps to enterprises on Databricks. With TIMESTAMP(9) and TIMESTAMP_NTZ(9), you can retain up to nine fractional-second digits in a native timestamp column and analyze high-frequency events without precision loss or workarounds that store timestamps as strings or integers. Timestamp nano support is fully open-sourced and integrated within Apache Spark™, Delta Lake, and Apache Iceberg.
The applications of Timestamp Nano extend to numerous domains. Enterprises can:
Timestamp Nano reduces friction throughout the lifecycle of high-frequency data, from ingestion and preparation to analysis. To demonstrate, consider a question from the head of quantitative research at a trading firm:
Did our strategy react to the quote that was actually available when each trade occurred?
The data needed to answer this question is a stream of market events. Quotes and trades arrive in a clear order, but several can occur inside one microsecond:
|
At microsecond precision, every event becomes 09:30:00.123456. The rows remain, but the sequence does not. Here is how Timestamp Nano preserves the information needed to answer the research question.
event_time as TIMESTAMP_NTZ(9), keeping the event time in the table schema rather than hiding it inside a string or integer.
|
|
The same pattern applies outside financial markets. A manufacturing team can preserve the order of sensor readings around a fault. A research team can reconstruct measurements captured during a fast-moving experiment. In each case, the nanosecond precision remains part of the data from source to analysis.
Timestamp Nano is available now in Beta. Get started today by reading the TIMESTAMP documentation and TIMESTAMP_NTZ documentation, and testing Timestamp Nano with your market, sensor, or scientific data.
If you have questions or feedback, please reach out to your Databricks account team.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.