The trace I originally shared was a HEAD request (GetObjectMetadataRequest), which only returns headers, so the actual denial reason was hidden and it just showed a bare "Forbidden". Re-running the same access as a List operation surfaces the real error message in the response body.

Running this in a serverless notebook:

dbutils.fs.ls("s3://<raw-data-bucket>")

gives:

ExecutionError: [UNAUTHORIZED_ACCESS] Unauthorized access: s3://<raw-data-bucket>: getFileStatus on s3://<raw-data-bucket>: shaded.databricks.awssdk.com.amazonaws.services.s3.model.AmazonS3Exception: Access to storage destination is denied because of serverless network policy; request: GET https://<raw-data-bucket>.s3-us-west-2.amazonaws.com / ... ListObjectsV2Request; Request ID: null, Extended Request ID: null ... credentials-provider: BasicSessionCredentials ... signature-present: true (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: null; Proxy: <redacted>)

[Trace ID: <redacted — can share via DM/ticket>]

So the denial is explicitly "because of serverless network policy" — it's coming from the egress proxy, not from AWS IAM. A few supporting facts:

- The workspace's only network policy is set to "Allow access to all destinations" and hasn't changed recently
- system.access.outbound_network is empty
- UC vends session credentials fine (signed request in the trace), and Browse in Catalog Explorer lists the bucket
- From the same serverless session, dbutils.fs.ls on the Databricks-auto-created external location works — only this manually created one is denied. Both buckets are in us-west-2, same region as the metastore

This looks like the egress proxy's storage allowlist being out of sync with the UC external locations for my workspace. Happy to share the trace ID and workspace ID privately so you can check which allowlist snapshot the proxy evaluated.