I want to consolidate the delta tables historical versions and write to another table in append mode
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 09:18 AM
Hi Team,
I have a table let's say, employee and it has 5 versions .
version 0 and 1 has column A, column B,
from version 2 column A has been changed to column C and column B to column D.
I want to ingest version by version without manual intervention. Can you please suggest the best practice here?
- Labels:
-
Workflows
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 10:55 AM
Hi @amitprasad01 , When you write to a new table, do you want to write the data from column A and column C to one single column A or you want to create two separate columns named A and C?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 10:59 PM
@Lakshay I want to create everything into latest columns that means column a will become column c and column b will become column D
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 11:04 AM
Do you want to ingest column a, column b and column a as c at a same time or you want to ingest version 1 then version 2 and so on??
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 12:17 PM
Have you checked out deltaTable.cloneAtVersion()?
