topic Re: S3 limited bucket permissions in Get Started Discussions

S3 limited bucket permissions

arnas — Thu, 22 May 2025 05:37:39 GMT

Hi,

can I run Databricks on limited/restricted S3 bucket folder, no access to bucket root level as it is restricted per project folder in IAM?

i.e s3://mybucket/myproject_abc/

Now I configured all permissions as per documentation
https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/storage-credentials

"Condition": {
"StringLike": {
"s3:prefix": "myproject_abc/*"
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mybucket/myproject_abc/*",
"arn:aws:s3:::mybucket"
],
"Sid": "AllowS3ActionsForProjectABC"
}

Re: S3 limited bucket permissions

SP_6721 — Thu, 22 May 2025 15:58:09 GMT

Hi @arnas

Yes, you can give Databricks access to just the S3 folder s3://mybucket/myproject_abc/ without exposing the whole bucket.

IAM Policy should include:

Bucket level: Allow s3:ListBucket on arn:aws:s3:::mybucket with a condition for the myproject_abc/ prefix.
Object level: Allow s3:GetObject, s3:PutObject, and s3:DeleteObject on arn:aws:s3:::mybucket/myproject_abc/*.

Re: S3 limited bucket permissions

arnas — Wed, 28 May 2025 13:56:14 GMT

aws cli works to list the folder, and I am able to download files from inside the folder
aws s3 ls
aws s3 cp

here is my policy, anything I am missing ?

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "AllowObjectActionsInCustomerPrefix",

"Effect": "Allow",

"Action": [

"s3:GetObject",

"s3:PutObject",

"s3:DeleteObject",

"s3:AbortMultipartUpload",

"s3:ListMultipartUploadParts"

"Resource": "arn:aws:s3:::mybucket/folder/*"

{

"Sid": "AllowListBucketInCustomerPrefix",

"Effect": "Allow",

"Action": [

"s3:ListBucket",

"s3:GetBucketLocation",

"s3:ListBucketMultipartUploads"

"Resource": "arn:aws:s3:::mybucket",

"Condition": {

"StringLike": {

"s3:prefix": "folder/*"

}

{

"Sid": "AllowAssumeRole",

"Effect": "Allow",

"Action": "sts:AssumeRole",

"Resource": "arn:aws:iam::MYACCOUNTID:role/mycustomer-databricks-access"

}

]

}

Re: S3 limited bucket permissions

arnas — Thu, 29 May 2025 06:33:15 GMT

Thanks, but no thanks, spam resides in JUNK folder