- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-04-2025 04:37 AM
Although not formal support, here are some things to consider as you troubleshoot the problem:
This issue sounds like it stems from a Unity Catalog external location configuration mismatch or permission issue, despite the external location itself being reachable. Here’s a structured checklist to help you troubleshoot the [TABLE_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST] error when creating a table:
✅
Things You’ve Already Verified
-
✅ Workspace is reachable and accessible
-
✅ External location is created and points to a valid container
-
✅ Networking on the Storage Account is open to all endpoints
-
✅ Volume creation on the same location works (implying access is technically fine)
-
✅ Table creation without an external location works
🔍
Troubleshooting Checklist
1. Check External Location and Storage Credential Mapping
-
-
The storage credential is of the correct type (e.g., managed identity or service principal).
-
It has permission (Storage Blob Data Contributor) on the container and the parent storage account.
Make sure the external location is correctly mapped to a valid storage credential, and that:
-
-
Run this SQL in a Databricks notebook in the new workspace to inspect:
DESCRIBE EXTERNAL LOCATION `<your_external_location_name>`;
2. Catalog Permissions
-
Ensure that your Unity Catalog catalog is correctly referencing the external location:
DESCRIBE CATALOG `<your_catalog_name>`;
-
-
-
USE CATALOG <catalog>
-
CREATE TABLE privilege on the catalog or schema level
Also check if the user (or the group they belong to) has both of these:
-
3. Check the Metastore Assignment
-
-
Go to Admin Console → Unity Catalog → Metastore assignments
-
Make sure the new workspace is correctly assigned and not in an inconsistent state
Confirm the workspace is attached to the correct Unity Catalog metastore:
-
4. Table Creation Command Structure
-
If you are doing a CREATE TABLE (non-managed), be sure you are not mixing paths or missing specifications. This can trip up UC-managed external catalogs.
-
Try creating the table with an explicit path override just to test:
CREATE TABLE my_catalog.my_schema.my_table (id INT)
LOCATION 'abfss://<container>@<account>.dfs.core.windows.net/<some_subpath>';
5. Schema Directory Issue
-
-
Manually create the schema directory, or try:
If the schema (aka “database”) directory does not yet exist in the storage account (under /catalog_name/schema_name/), Databricks may fail silently or with confusing errors.
-
CREATE SCHEMA my_catalog.my_schema;
6. Unity Catalog Table Path Validation
-
-
Try checking the table metadata in UC using:
Unity Catalog uses a specific metadata path structure. If the UC metastore cannot reconcile the metadata with the storage path, it may show RESOURCE_DOES_NOT_EXIST.
-
SHOW TABLES IN my_catalog.my_schema;
🛠️ Additional Recommendations
-
Try recreating the external location using the Databricks UI and make sure it’s validated.
-
Double check the workspace identity (Managed Identity or Service Principal) is the one configured in the storage credential and has access to the container.
-
If all else fails: Try creating the same setup in a test workspace with minimal security configs to rule out weird propagation or policy issues.
🧩 Root Cause Theories
Based on your description, here are the likely culprits:
-
Unity Catalog is unable to register or recognize the metadata for the new table due to a missing schema directory or improper mapping.
-
The external location credential identity has storage access but not metastore access.
-
The workspace might not have fully propagated Unity Catalog setup after being added to the metastore.
Hope this sets you in the right direction.
Cheers, Lou.