when we are trying to create folder/file or list file using dbutils we are getting forbidden error in aws

[karthik_p]

HI Team,

we have created new premium workspace with custom managed vpc, workspace deployed successfully in AWS. we are trying to create folder in dbfs, we are getting below error. we have compared cross account custom managed role (Customer-managed VPC with custom policy restrictions), but still we are getting "GetObjectMetadataRequest" forbidden. i am not seeing issue in terms of permissions, did anything had been updated from databricks metastore end .

Caused by: java.nio.file.AccessDeniedException: s3a://xxxxx/nvirginia-prod/xxxx.meta/mounts: 

getFileStatus on s3a://xxxxx nvirginia-prod/xxxx.meta/mounts: 

com.amazonaws.services.s3.model.AmazonS3Exception:

Forbidden; request: HEAD https://xxxx nvirginia-prod/xxxxx.meta/mounts {} Hadoop 2.7.7, aws-sdk-java/1.12.261 Linux/5.4.0-1096-aws OpenJDK_64-Bit_Server_VM/25.342-b07 java/1.8.0_342 scala/2.12.15 kotlin/1.6.21 vendor/Private_Build cfg/retry-mode/legacy com.amazonaws.services.s3.model.GetObjectMetadataRequest; Request ID: xxxx, Extended Request ID: xxxxx, Cloud Provider: AWS, Instance ID: i-xxxx 

(Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: xxxxx; S3 Extended Request ID: xxxxx=; Proxy: null)

, S3 Extended Request ID: xxxxxxx=:403 Forbidden

[Debayan]

Hi, The HTTP 403 Forbidden response status code indicates that the server understands the request but refuses to authorize it. Could you please check the S3 permissions and other related permissions in IAM?

Also, please tag @Debayan​ with your next response which will notify me. Thank you!

[karthik_p]

@Debayan Mukherjee​ yes, we have compared IAM cross account role and bucket permissions, everything looks good as per databricks recommendations . we have picket custom vpc policy and applied that

[karthik_p]

@Debayan Mukherjee​ client is still in free version, within 3 days it will be converted to premium. Is above issue because of free tier

[Debayan]

@karthik p​ Can you please check and confirm now?

[karthik_p]

@Debayan Mukherjee​ Issue resolved, looks cloud team have not updated required security groups that has been shared, after revisiting them we are able to find missing security groups and added them

[Debayan]

@karthik p​ , Thanks for the confirmation. Glad to know!