Hi @emorgoch 

This is a known pain point. Here's what's happening and how to resolve it:

Root Cause

When you created the catalog/schema/table using that container as managed storage, Unity Catalog registered that path internally as a managed storage location. Even after you:

• Dropped the catalog
• Deleted the __unitystorage directory from ADLS
• Force-deleted and re-created the external location

...the metastore still retains metadata about that path being associated with managed storage. This metadata persists for at least 7 days (the soft-delete recovery window), and in some cases can become "orphaned" if the catalog was dropped before the tables underneath were fully purged.

The LOCATION_OVERLAP error occurs because UC prevents external locations (or operations like AutoLoader READ FILES) from accessing paths that overlap with known managed storage locations — even if the actual data is gone.

Solutions (in order of preference)

1. Wait for the 7-day purge window

After dropping a catalog with managed tables, UC retains soft-delete metadata for 7 days. After the purge completes (~7 days + up to 48h for final cleanup), the managed storage reference should be released.

Check: How long ago did you drop the catalog? If less than 7-9 days, wait it out.

2. Contact Databricks Support (recommended if >9 days)

If the 7-day window has passed and you're still seeing LOCATION_OVERLAP, the managed storage reference is likely orphaned in the metastore metadata. This can happen when:

• The catalog was dropped with CASCADE but the purge didn't fully complete
• The __unitystorage directory was manually deleted from storage before UC could clean up
• The external location was force-deleted while managed table metadata still existed

Databricks Support can manually remove the orphaned managed storage reference from your metastore's internal metadata. Provide them:

• The external location name: planview-adaptivework_psinkext-udp_extloc
• The storage URL (the abfss://...dfs.core.windows.net/ path)
• The error message with RequestId
• Confirmation that the catalog and all its contents are intentionally deleted

3. Workaround: Use a subdirectory-scoped external location

If you need to unblock immediately while waiting for Support:

Instead of pointing the external location at the container root:

abfss://container@account.dfs.core.windows.net/

Create the external location pointing to a subdirectory where your files actually live:

abfss://container@account.dfs.core.windows.net/data/

Or move your landing files into a subfolder. The overlap check is path-based — a subdirectory that doesn't collide with the old __unitystorage path (which was at the root level) should work.

4. Verify there are no hidden references

Run these queries to check for any lingering references in your metastore:

-- Check if any catalogs still reference this storage location
SELECT * FROM system.information_schema.catalog_tags 
WHERE catalog_name LIKE '%<your_catalog_name>%';

-- Check all external locations for overlapping paths
SHOW EXTERNAL LOCATIONS;

-- Look for managed storage locations at the metastore level
DESCRIBE METASTORE;

Also verify with the Unity Catalog REST API:

GET /api/2.1/unity-catalog/external-locations

to see if there's a hidden managed storage location you're not seeing in the UI.

Prevention for the future

1. Never use a container root as managed storage if you also plan to use it for external data/AutoLoader. Use a dedicated container or a dedicated subdirectory for managed storage.
2. Don't manually delete __unitystorage from storage — let UC manage the lifecycle. Manual deletion can orphan metadata.
3. Drop tables/schemas before dropping the catalog (rather than CASCADE) if you want finer control over cleanup.
4. Separate concerns: Use one container/path for managed tables and a different one for landing files/external data.

TL;DR

The LOCATION_OVERLAP persists because UC's metastore still has an internal reference marking that container root as "managed storage" — even after you deleted everything visually. The force-delete of the external location + physical deletion of __unitystorage didn't clear the metastore's managed-storage registry.

Quick fix: Point external location to a subdirectory instead of root. Permanent fix: Wait 7-9 days for purge, or open a Databricks Support ticket to manually remove the orphaned managed storage reference.