cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

when we are trying to create folder/file or list file using dbutils we are getting forbidden error in aws

karthik_p
Esteemed Contributor

HI Team,

we have created new premium workspace with custom managed vpc, workspace deployed successfully in AWS. we are trying to create folder in dbfs, we are getting below error. we have compared cross account custom managed role (Customer-managed VPC with custom policy restrictions), but still we are getting "GetObjectMetadataRequest" forbidden. i am not seeing issue in terms of permissions, did anything had been updated from databricks metastore end .

Caused by: java.nio.file.AccessDeniedException: s3a://xxxxx/nvirginia-prod/xxxx.meta/mounts: 

getFileStatus on s3a://xxxxx nvirginia-prod/xxxx.meta/mounts: 

com.amazonaws.services.s3.model.AmazonS3Exception:

Forbidden; request: HEAD https://xxxx nvirginia-prod/xxxxx.meta/mounts {} Hadoop 2.7.7, aws-sdk-java/1.12.261 Linux/5.4.0-1096-aws OpenJDK_64-Bit_Server_VM/25.342-b07 java/1.8.0_342 scala/2.12.15 kotlin/1.6.21 vendor/Private_Build cfg/retry-mode/legacy com.amazonaws.services.s3.model.GetObjectMetadataRequest; Request ID: xxxx, Extended Request ID: xxxxx, Cloud Provider: AWS, Instance ID: i-xxxx 

(Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: xxxxx; S3 Extended Request ID: xxxxx=; Proxy: null)

, S3 Extended Request ID: xxxxxxx=:403 Forbidden

6 REPLIES 6

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, The HTTP 403 Forbidden response status code indicates that the server understands the request but refuses to authorize it. Could you please check the S3 permissions and other related permissions in IAM?

Also, please tag @Debayan​ with your next response which will notify me. Thank you!

karthik_p
Esteemed Contributor

@Debayan Mukherjee​ yes, we have compared IAM cross account role and bucket permissions, everything looks good as per databricks recommendations . we have picket custom vpc policy and applied that

karthik_p
Esteemed Contributor

@Debayan Mukherjee​ client is still in free version, within 3 days it will be converted to premium. Is above issue because of free tier

Debayan
Esteemed Contributor III
Esteemed Contributor III

@karthik p​ Can you please check and confirm now?

karthik_p
Esteemed Contributor

@Debayan Mukherjee​ Issue resolved, looks cloud team have not updated required security groups that has been shared, after revisiting them we are able to find missing security groups and added them

Debayan
Esteemed Contributor III
Esteemed Contributor III

@karthik p​ , Thanks for the confirmation. Glad to know!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!