AccessDenied error on s3a:// bucket due to Serverless Network Policy in Databricks SQL Endpoint

jeremylllin
New Contributor

‎06-20-2025 03:06 AM

I wrote this code in Notebook

files = dbutils.fs.ls("s3a://testbuket114/")

for f in files:
print(f.name)
 
it caused err
s3a://testbuket114/: getFileStatus on s3a://testbuket114/: com.amazonaws.services.s3.model.AmazonS3Exception: Access to storage destination is denied because of serverless network policy; request: GET http://testbuket114.s3.us-east-1.amazonaws.com {key=[], key=[false], key=[2], key=[2], key=[/]} Hadoop 3.3.6, aws-sdk-java/1.12.638 Linux/5.15.0-1072-aws OpenJDK_64-Bit_Server_VM/17.0.13+11-LTS java/17.0.13 scala/2.12.15 kotlin/1.9.10 vendor/Azul_Systems,_Inc. cfg/retry-mode/legacy com.amazonaws.services.s3.model.ListObjectsV2Request; Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: unknown credentials-provider: com.amazonaws.auth.BasicSessionCredentials credential-header: AWS4-HMAC-SHA256 Credential=ASIA2OAJT3OJXLJL4HDN/20250620/us-east-1/s3/aws4_request signature-present: true (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: null; S3 Extended Request ID: null; Proxy: 192.168.200.20), S3 Extended Request ID: null:AccessDenied SQLSTATE: 42501

err in sql query

INTERNAL: [UNAUTHORIZED_ACCESS] Unauthorized access: s3a://testbuket114/: getFileStatus on s3a://testbuket114/: com.amazonaws.services.s3.model.AmazonS3Exception: Access to storage destination is denied because of serverless network policy; request: GET http://testbuket114.s3.us-east-1.amazonaws.com {key=[], key=[false], key=[2], key=[2], key=[/]} Hadoop 3.3.6, aws-sdk-java/1.12.638 Linux/5.15.0-1072-aws OpenJDK_64-Bit_Server_VM/17.0.13+11-LTS java/17.0.13 scala/2.12.15 kotlin/1.9.10 vendor/Azul_Systems,_Inc. cfg/retry-mode/legacy com.amazonaws.services.s3.model.ListObjectsV2Request; Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: unknown credentials-provider: com.amazonaws.auth.BasicSessionCredentials credential-header: AWS4-HMAC-SHA256 Credential=REDACTED_ACCESS_KEY(da3c912f)/20250620/us-east-1/s3/aws4_request signature-present: true (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: null; S3 Extended Request ID: null; Proxy: 192.168.200.20), S3 Extended Request ID: null:AccessDenied SQLSTATE: 42501

 

I already create external location in catalog

jeremylllin_0-1750413768379.png

and credential 

jeremylllin_1-1750413858121.png

Despite all this, I still get the same error with no Request ID or Extended Request ID.

Has anyone encountered this issue or have suggestions on what else I should check or configure? Could there be caching or propagation delays in the Serverless Network Policy? Or any other hidden settings that might block access?

Thanks in advance for your help!

 

0 Kudos