Hi @yawningbrain,
You can't repoint the metastore root in place, so the supported path is to give every catalog its own managed storage on the new ADLS and then remove the metastore root. Here is the sequence with the relevant docs.
Step 1 - Give your business catalogs explicit managed storage on the new ADLS. First, find which catalogs still fall back to the metastore root. Run "DESCRIBE CATALOG EXTENDED <catalog_name>" and look at the Storage Root field. Any catalog with no catalog-level location is relying on the root.
For each of those, register an external location for the new ADLS, backed by a storage credential with an Azure managed identity, then set the catalog location.
ALTER CATALOG my_catalog SET MANAGED LOCATION 'abfss://my-container@mynewstorage.dfs.core.windows.net/my_catalog';
This needs Databricks SQL or Databricks Runtime 18.1 and above, and you need CREATE MANAGED STORAGE on the external location plus ownership or the MANAGE privilege on the catalog. The managed location string must be 150 characters or fewer. You can refer to these links. ALTER CATALOG and Specify a managed storage location.
Altering the managed location only affects new managed tables and volumes created after the change. Existing managed objects stay where they are and keep working, so your pipelines will not break. Worth knowing too that storage precedence runs schema location over catalog location over metastore location, so removing the root only touches catalogs that have no catalog-level location.
Step 2 - Remove the metastore root. Once your business catalogs have explicit storage, go to the account console, open Catalog, select your metastore, open the Configuration tab, and click Remove next to the ADLS Gen 2 path. What happens next is documented here. Catalogs that already have their own root are unaffected. Catalogs without one are given the old metastore root path as their catalog-level managed storage, which the docs call push-down, and access continues without interruption. Depending on how your metastore was created, a new external location named prior_metastore_root_location and an associated storage credential may be auto-created for the old path. After this, every new catalog must specify its own location.
If any catalog must keep using the exact old path after removal, you can pre-register that path as an external location and set it explicitly before you remove the root. That gives you a named external location and credential you control instead of the auto-generated prior_metastore_root_location.
Step 3 - The system and __databricks_internal catalogs. These two need separate consideration, and the removal doc above does not actually describe either of them. It only covers the generic push-down rule for ordinary catalogs, so treat these as follows.
system holds your account's operational system-table data, and that data is stored in a Databricks-hosted storage account in the same region as your metastore, shared with you through open sharing. It does not physically live in your metastore root. So even if a legacy path shows up in catalog properties after push-down, that alone does not mean the system data sits in the old account, and retiring the old ADLS should not break system tables.
__databricks_internal is a platform-managed catalog used for pipeline materialisations such as materialised views and streaming tables, and Databricks advises against accessing its tables directly. Unlike system, this data can genuinely reside in customer-managed storage, so this is the one that could still tie you to the legacy account. The removal behaviour for it is not documented, so I would have Databricks Support confirm the migration or cleanup procedure before changing any internal paths or retiring the storage account.
Removing the root does not get you off the legacy ADLS. Push-down only rewrites the pointer. It moves no data. So root removal is safe and low-impact, but it is not the step that lets you decommission the legacy container.
Before assuming zero dependency, confirm each business catalog actually has its data on the new ADLS. If a business table was a managed table under the old root, its files may still sit in the old container even though the table reads fine. The thing to check is whether you have any Unity Catalog pipelines, materialised views, or streaming tables, since those materialisations land in __databricks_internal and are the tangible remaining tie to the old storage. For a full cutover later, deep-clone or CTAS the remaining managed tables into the new locations. Keep the old account and its access permissions intact until those dependencies are resolved, because the documented root-removal behaviour preserves read access through a repointed pointer, it is not a guarantee that the underlying storage is safe to delete.
This should ideally be zero-impact for running pipelines. Existing managed tables keep their data in the original storage location regardless of changes to the metastore or catalog root. Only newly created managed tables and volumes use the updated location. External tables are unaffected since they reference their own storage paths.
If you use open sharing and the metastore root was used as shared notebook storage, you must remove the notebook from the share and re-add it using a dedicated storage location before you can remove the root.
Hope this helps.
If this answer resolves your question, could you mark it as โAccept as Solutionโ? That helps other users quickly find the correct fix.
Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***
