topic Re: Permission Denied when Creating External Tables Using Workspace Default Credential in Data Engineering

Permission Denied when Creating External Tables Using Workspace Default Credential

j_unspeakable — Sun, 15 Jun 2025 14:34:01 GMT

I’m building out schemas, volumes, and external Delta tables in Unity Catalog via Terraform. The schemas and volumes are created successfully, but all external tables are failing.

The error message from Terraform doesn't highlight what the issue is but returns just (Error: cannot create sql table: statement failed to execute: FAILED)

I tested manually in the Databricks SQL editor with:

CREATE TABLE catalog.schema.fr012 USING DELTA LOCATION 'abfss://nexus-dev@stinfinasdevuks.dfs.core.windows.net/gold/table/fr002';

and I got this error:

PERMISSION_DENIED: The credential 'dbw_infinas_dev_uks' is a workspace default credential that is only allowed to access data in the following paths: 'abfss://unity-catalog-storage@dbstoragefkilqgly2oevu.dfs.core.windows.net/1390520157504627'. Please ensure that any path accessed using this credential is under one of these paths.

However, I was able to create and browse the same External Location in the Unity Catalog UI using the same default credential.

I did some research and found that workspace default credentials in Databricks are not exclusively scoped to Databricks' managed storage paths. While they can be used to access managed storage, they can also be used to access external storage locations as well, particularly when leveraging storage credentials and external locations.

Questions:

Is it expected that the workspace default credential can access an External Location but not be allowed to use it in external table creation?
Will creating a separate storage credential and external location (backed by an SPN/MANAGED_IDENTITY) resolve the table-creation issue?
I found examples stating default credentials can access external data—are these outdated, or am I missing steps?

I’ve attached screenshots of:

The External Location with the default credential
The PERMISSION_DENIED error message

Re: Permission Denied when Creating External Tables Using Workspace Default Credential

j_unspeakable — Mon, 16 Jun 2025 14:42:58 GMT

I was able to fix this by creating a new Access Connector for Azure Databricks, assigning the appropriate permission to the storage account, creating a new storage credential and using the credential to register my external location.

https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/cloud-storage/storage-credentials

Re: Permission Denied when Creating External Tables Using Workspace Default Credential

IanB — Thu, 04 Sep 2025 15:26:48 GMT

I had the same issue, thank you my dear

Re: Permission Denied when Creating External Tables Using Workspace Default Credential

artopihlaja — Mon, 13 Oct 2025 18:51:42 GMT

Feature or bug, I discovered the same. I couldn't create tables with the default credential.
To test, I assigned the default credential and a custom credential the same access rights to the storage container that is the target of the external location. When the external location was assigned to the the default credential, I got the error message PERMISSION_DENIED: The credential 'tekdata_databricks_dev' is a workspace default credential that is only allowed to

When I reassigned the external location to the custom credential, table creation was OK. I didn't find this behaviour in the documentation, but Copilot pulled it up from somewhere.

Re: Permission Denied when Creating External Tables Using Workspace Default Credential

almed — Mon, 15 Dec 2025 16:12:20 GMT

Thanks j_unspeakable, and more thanks to artopihlaja for "I reassigned the external location to the custom credential", because that was the dealbreaker for me.