Is it possible to retain original deltatable data with Unity Catalog?

[Yuki]

Hi everyone,

I have a question regarding data retention in Unity Catalog. In the pre–Unity Catalog setup, I believe that even if we dropped an external table, the underlying data files remained intact.

However, in the current best practices for Unity Catalog, external tables are no longer recommended in most case.
https://docs.databricks.com/aws/en/data-governance/unity-catalog/best-practices#-managed-and-externa... 

Additionally, I noticed that the documentation for the UNDROP command mentions a 7-day retention period.
https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-undrop-table 
Previously, we could retain the actual data files permanently, but is that no longer feasible under Unity Catalog?

Any insights or experiences would be greatly appreciated!

Thanks in advance,

[mani_22]

Hi @Yuki,  If you drop an external table, the underlying data remains accessible even now. Only the table definition is removed from the metastore, while the actual data is retained. The UNDROP command for an EXTERNAL table simply recreates the table in the metastore, since the data is still available. It's similar to running the CREATE TABLE command with the LOCATION specified.

Using managed tables is recommended because they benefit from several automatic optimisations, such as Auto Compaction, Auto Optimize, Faster metadata reads (through metadata caching), Intelligent file size optimization