- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-24-2025 06:11 AM
Hi @hobrob ,
This is such a cool idea and honestly exactly the kind of thing that helps people crossing the bridge from Teradata to Databricks without losing their sanity on temporal logic every time 😄
Those type 2/4/6 SCD and period-comparison helpers are pain points in almost every migration, so having a reusable, UC-backed set of functions is genuinely valuable, especially for teams who are still thinking in Teradata patterns but want to “land well” on the Lakehouse.
A few thoughts / ideas from my side:
It would be great to see a couple of end-to-end examples:
classic SCD2 customer dimension
overlapping policy / tariff periods
“find gaps” / “find overlaps” style queries using your UDFs
Long-term, this could easily become a mini “Teradata to Databricks” toolkit:
temporal helpers (what you’ve started)
compatibility functions/operators people miss most
maybe even some “Teradata to Spark SQL” patterns documented.
On the CI/CD side, even a simple GitHub Actions pipeline that runs unit tests against a small Delta test dataset would be a great reference for others starting out.
Keep up with the great work!