RESTORE deletes part of the delta table's history
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-21-2025 06:14 AM
Having a delta table with the history of 15 versions (see screenshot). After running the command:
RESTORE TABLE hive_metastore.my_schema.my_table TO VERSION AS OF 6;
And then running DESCRIBE HISTORY (see screenshot) it seems that a new version (RESTORE) was added, but the first 10 versions are gone. I checked this with more tables and the same thing happened. Why is that? Is this expected behaviour?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-21-2025 10:39 AM
it's not. I haven't observed this behavior. According to the delta lake documentation "Using the restore command resets the table’s content to an earlier version, but doesn’t remove any data. It simply updates the transaction log to indicate that certain files should not be read. "
