topic Re: Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day. in Data Engineering

Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.

vinaykumar — Tue, 14 Feb 2023 16:03:21 GMT

Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.

For example , let’s assume we are doing multiple operation on table in a day every minute and want to keep time travel history for each day of latest version of a day for one week .

Re: Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.

Lakshay — Tue, 14 Feb 2023 17:52:40 GMT

If I understood correctly, you want to keep only one version per day regardless of the number of transactions done on that day. The problem in achieving this is that if you delete any of the versions in a delta table, you will lose the ability to time travel to any version before that.

The only way I think this is achievable is by keeping some sort of record(in a separate table perhaps) of the latest version for each day and use information from there to do a time travel

Re: Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.

Anonymous — Fri, 17 Feb 2023 05:48:28 GMT

Hi @vinay kumar

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.

vinaykumar — Sat, 18 Feb 2023 19:38:51 GMT

Yes , Issue got resolved . Thanks