cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

APPLY INTO Highdate instead of NULL for __END_AT

c-thiel
New Contributor

I really like the APPLY INTO function to keep track of changes and historize them in SCD2.
However, I am a bit confused that current records get an __END_AT of NULL. Typically, __END_AT should be a highgate (i.e. 9999-12-31) or similar, so that a point in time query can simply use:

SELECT * FROM TABLE where '<my-date>' BETWEEN __START_AT and __END_AT

Right now I believe I have to use COALESCE on __END_AT to get the correct results. Is there an option to set a highgate instead of NULL for __END_AT?

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group