Hello everyone,
I am looking for clarification about the 14-day rollback period after converting an external table to a managed table using: ALTER TABLE table_name SET MANAGED.
Databricks documentation states that the conversion can be rolled back within 14 days. It also says that, after this period, Predictive Optimization or a manual VACUUM can delete data from the old external location to avoid unnecessary storage costs.
I would appreciate clarification on the following scenarios which are not documented (afaik):
1- Running VACUUM before day 14
If I run VACUUM before the 14-day rollback period ends, does it affect only files in the new managed-table location? Or can it also delete files from the original external location?
2- Retention period shorter than 14 days
If delta.deletedFileRetentionDuration is set to less than 14 days, for example 7 days: Does the rollback window become 7 days?
If I run VACUUM on day 8, could it delete data needed for rollback, or it will just affects the data in the managed location?
3- Retention period longer than 14 days
If delta.deletedFileRetentionDuration is set to more than 14 days, for example 20 days. Does the rollback window remain 14 days or it extends to 20 days?
My assumptions are that the rollback period is fixed to 14 days so, if you run vacuum before that period will not touch the data in the external location no matter the retention-days. If the retention is more than 20 days and if I run a vacuum or PO kicks in, the data in old location will be deleted.
Has anyone tested these cases or received clarification from Databricks? I could not find details about how the rollback period interacts with delta.deletedFileRetentionDuration, manual VACUUM, and Predictive Optimization.
Thanks for any guidance.
