Unity Catalog external location access error after metastore attachment

[Khasim_1]

Hi everyone,

I am running into a permission issue when trying to query data from an external S3 storage location managed through Unity Catalog.

Here are the symptoms:

• The Setup: Storage credentials and an external location were successfully created, and the metastore is properly linked to the workspace.
• The Problem: When running a query or trying to list files in the external location, the job fails with a permission denied error, even though the assigned IAM role appears correct on the cloud side.
• What I'm looking for: Has anyone faced a similar authorization gap between AWS IAM and Unity Catalog storage credentials? What troubleshooting steps or logs should I check next?

Any guidance from the community would be greatly appreciated!

[data_pulse]

We hit something similar. In our case the job ran as a service principal that was missing the required UC privilege, even though the IAM role was fine. For path-based access that’s READ FILES on the external location, for tables it’s SELECT, USE CATALOG/USE SCHEMA and for volumes it’s READ VOLUME. Granting the right one fixed it.

Few common causes and triage steps:

• UC privileges: Confirm the job’s identity and its grants, and that the path falls under the external location

   SHOW GRANTS ON EXTERNAL LOCATION my_location;
   DESCRIBE EXTERNAL LOCATION my_location;​

if privilege is missing, then

GRANT READ FILES ON EXTERNAL LOCATION my_location TO `user_or_sp`​

• Compute: Make sure the job runs on UC-enabled compute (standard/ dedicated access mode / serverless).
• Validation: Run Validate Configuration on the storage credential and Test connection on the external location in Catalog Explorer, then test directly. Also try:

DESCRIBE STORAGE CREDENTIAL my_credential;
LIST 's3://my-bucket/path/';​

• IAM and AWS: Check the trust policy and external ID (including self-assuming role), S3 permissions and KMS key policy if encrypted.
• Workspace binding: Check the external location’s bindings, especially if the metastore association changed recently via Catalog Explorer -> Connect -> External Location or with Databricks CLI

databricks external-locations get my_location

databricks workspace-bindings get-bindings external-location my_location​

[pradeep_singh]

A correct IAM role is necessary but not sufficient. Access to an external location goes through two separate checks, and the wording of the error tells you which one failed. Here's how to find it quickly.

Step 0 – Capture the exact error. Run a direct path read from the same compute and identity that fails:

LIST 's3://<bucket>/<prefix>/';

Then match the message:

Error text containsFailing layerGo to

PERMISSION_DENIED: User does not have READ FILES on External Location ... (or USE CATALOG / USE SCHEMA / SELECT)	Unity Catalog privileges	Step 1
"not accessible in the current workspace" / location greyed out in Catalog Explorer	Workspace binding	Step 2
"AWS IAM role does not have READ permissions on url ...", "403 Forbidden from cloud storage provider", "Failed on assuming role"	AWS (trust, S3, KMS, bucket policy)	Step 4
Errors about the Unity API token, or access mode	Compute	Step 3

Step 1 – Unity Catalog privileges (the most common cause). A storage credential does not grant users anything. The calling principal needs privileges on the external location, and for tables, on the catalog, schema and table too. For a job, that means the job's run-as identity.

SHOW GRANTS ON EXTERNAL LOCATION `my_location`;

GRANT READ FILES  ON EXTERNAL LOCATION `my_location` TO `data_engineers`;

GRANT WRITE FILES ON EXTERNAL LOCATION `my_location` TO `data_engineers`;  -- writes need READ FILES too

-- for external tables / volumes on that path:

GRANT USE CATALOG ON CATALOG my_cat TO `data_engineers`;

GRANT USE SCHEMA  ON SCHEMA  my_cat.my_sch TO `data_engineers`;

GRANT SELECT ON TABLE my_cat.my_sch.my_tbl TO `data_engineers`;   -- or READ VOLUME on a volume

Notes:

• A principal with WRITE FILES but no READ FILES still gets PERMISSION_DENIED on writes.
• Databricks recommends granting READ VOLUME on a volume rather than READ FILES on raw paths for most users.

Step 2 – Workspace bindings. If the location (or its catalog) is set to isolated, it only works from bound workspaces.

databricks external-locations get my_location            # check isolation_mode

databricks workspace-bindings get-bindings external-location my_location

Either add your workspace ID, or tick "All workspaces have access" on the location's Workspaces tab.

Step 3 – Compute. Use a UC-capable access mode:

• Standard (formerly Shared) or Dedicated (formerly Single user) clusters, a SQL warehouse, or serverless.
• "No isolation shared" and legacy clusters can't use UC external locations.

Step 4 – AWS side. Start with Catalog > External data > Credentials > your credential > Validate configuration, then Test connection on the external location. The results show which step fails: self-assume, read, list, write or delete. Then check, in this order:

1. Trust policy: it must trust the Databricks UC master role ARN shown in the docs or on the credential page, and the role itself (self-assuming), with sts:ExternalId equal to the external ID shown on the credential page. Databricks blocks non-self-assuming roles. If you recreated the role, re-copy the external ID.
2. Permissions policy: s3:ListBucket and s3:GetBucketLocation on arn:aws:s3:::<bucket>, and object actions (GetObject, plus PutObject/DeleteObject for writes) on arn:aws:s3:::<bucket>/<prefix>/*. A common mistake is scoping ListBucket to the /* object ARN.
3. KMS (SSE-KMS buckets): the role needs kms:Decrypt/GenerateDataKey/Encrypt, and the KMS key policy must name the role.
4. Bucket policy explicit Deny: look for conditions such as aws:SourceVpce, aws:SourceVpc, aws:PrincipalOrgID or encryption headers.
• An explicit Deny beats every Allow.
• UC metadata and validation calls can come from the Databricks control plane rather than your workspace VPC endpoint. A "deny unless my VPCE" policy can therefore break UC even when your clusters reach S3.
• If the policy requires encryption headers, set the encryption algorithm on the external location.
5. SCPs and the bucket ACL: an Organizations SCP can deny S3 or KMS actions. Also, the docs require that the bucket has no S3 ACL attached; use Object Ownership = Bucket owner enforced.

Logs to check:

• AWS CloudTrail: filter on the role ARN, looking at AssumeRole events and S3/KMS AccessDenied events. The errorMessage usually names the denying policy type (identity, resource, SCP or VPC endpoint). S3 data events must be enabled to see object-level denies.
• Databricks: system.access.audit with service_name = 'unityCatalog' shows which principal and which action UC evaluated:

SELECT event_time, user_identity.email, action_name, request_params, response.status_code, response.error_message

FROM system.access.audit

WHERE service_name = 'unityCatalog'

  AND event_date >= current_date() - 1

  AND response.status_code <> 200

ORDER BY event_time DESC;

If Validate configuration passes, the bucket policy has no Deny, and you still get a 403, open a support ticket with the request ID from the error. That case needs a server-side look at the assume-role attempt.