cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

External to Managed

adeosthali
Databricks Partner
We are looking to migrate to managed tables using ALTER TABLE fq_table_name SET MANAGED
 
During migration process we need to have ability to switch between external & managed tables & vice versa.
UNSET MANAGED works for 14 days.
 
But I'm unable to just delete the converted managed table & rebuild as external exact same external location.
Once table is deleted spark.catalog.tableExists(fq_table_name) returns false as expected.
While trying to recreate it using below code returns error that table does not exist instead of creating table.
                df_final.write.format("delta")
                .mode("append")
                .option("path","Original_External_Storage_Path")
                .saveAsTable(runtime_contract.entity_full_name)
 
same process works if the table starts its life as managed.
i.e. I can have managed table x deleted & created as external.



1 ACCEPTED SOLUTION

Accepted Solutions

anshu_roy
Databricks Employee
Databricks Employee

Hello,

Thanks for sharing your investigation.

You’re correct: you can’t immediately recreate the converted table as an external table on the same original path after dropping it. The Unity Catalog table object remains in a soft‑deleted state for 7 days and is only fully removed after that period, so the underlying path remains reserved for the SET MANAGED / UNSET MANAGED lifecycle (with a rollback window of up to 14 days).

During that window you must either:

  • Use ALTER TABLE ... UNSET MANAGED to roll back to the original external table, or

  • Create a new external table on a different path, or recreate it as a managed table instead.

You can find the official documentation here:
https://learn.microsoft.com/azure/databricks/tables/convert-external-managed

View solution in original post

1 REPLY 1

anshu_roy
Databricks Employee
Databricks Employee

Hello,

Thanks for sharing your investigation.

You’re correct: you can’t immediately recreate the converted table as an external table on the same original path after dropping it. The Unity Catalog table object remains in a soft‑deleted state for 7 days and is only fully removed after that period, so the underlying path remains reserved for the SET MANAGED / UNSET MANAGED lifecycle (with a rollback window of up to 14 days).

During that window you must either:

  • Use ALTER TABLE ... UNSET MANAGED to roll back to the original external table, or

  • Create a new external table on a different path, or recreate it as a managed table instead.

You can find the official documentation here:
https://learn.microsoft.com/azure/databricks/tables/convert-external-managed