cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Access denied error to s3 bucket in Databricks notebook

phoebe_dt
New Contributor

When running a databricks notebook connected to an s3 cluster I randomly but frequently experience the following error:

 

java.nio.file.AccessDeniedException: s3://mybucket: getFileStatus on s3://mybucket: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden; request: HEAD https://mybucket.s3.eu-west-2.amazonaws.com hybrid {} Hadoop 3.3.4, aws-sdk-java/1.12.189 Linux/5.15.0-1045-aws OpenJDK_64-Bit_Server_VM/25.362-b09 java/1.8.0_362 scala/2.12.14 vendor/Azul_Systems,_Inc. cfg/retry-mode/legacy com.amazonaws.services.s3.model.GetObjectMetadataRequest; Request ID: BPE1N5PBRHX6AXXX, Extended Request ID: A9yWOO63cwg7PgdVihRC/bqVlOBwElgqYliThPpm56lH0lM/Xf09+g8Dkzdylpp422togli3000=, Cloud Provider: AWS, Instance ID: i-0ed293c3b4d8b0e15 credentials-provider: com.amazonaws.auth.AnonymousAWSCredentials credential-header: no-credential-header signature-present: false (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: BPE1N5PBRHX6AXXX; S3 Extended Request ID: A9yWOO63cwg7PgdVihRC/bqVlOBwElgqYliThPpm56lH0lM/Xf09+g8Dkzdylpp422togli3000=; Proxy: null), S3 Extended Request ID: A9yWOO63cwg7PgdVihRC/bqVlOBwElgqYliThPpm56lH0lM/Xf09+g8Dkzdylpp422togli3000=:403 Forbidden

 

When submitting jobs to the cluster using vs code I don't have this issue. Is there a best practice for setting up s3 in notebooks? The error is intermittent but happens frequently without any obvious cause and interrupts development.

Any help is greatly appreciated!

2 REPLIES 2

Kaniz_Fatma
Community Manager
Community Manager

Hi @phoebe_dt , The error message you're seeing indicates that the Databricks Notebook is trying to access your S3 bucket but is being denied. This is likely due to the AWS credentials not being properly set up or accessible in the notebook environment.

Kaniz_Fatma
Community Manager
Community Manager

Hi @phoebe_dt , Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!