Allow read access to S3 buckets from one AWS accounts to other AWS accounts.

[164079]

Dear team, 

We have several AWS accounts with S3 buckets, the databricks setup is on our dev AWS account and we would like to allow instance profile to have read permission on all our S3 buckets on the other AWS accounts ( without using bucket policy which require us to add it on any bucket)

I am trying using the assume role but dosent work, getting access denied.

It is working only if i set S3 bucket permissions on my other/remote AWS account bucket policy.

Please advise

Thansks!

[User16255483290 · Accepted solution]

Can you please share the IAM role policy in the secondary account (Bucket account) ?

Just wanted to know have you tried setting the config in the cluster.

spark.hadoop.fs.s3a.bucket.<s3-bucket-name>.aws.credentials.provider org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider

spark.hadoop.fs.s3a.bucket.<s3-bucket-name>.assumed.role.arn arn:aws:iam::<bucket-owner-account-id>:role/Master_Role

[164079]

Thank you @D Raj Kumar​ 

Added it and now its works!

Thanks