S3 limited bucket permissions

arnas
New Contributor II

‎05-21-2025 10:37 PM

Hi,

can I run Databricks on limited/restricted S3 bucket folder, no access to bucket root level as it is restricted per project folder in IAM?

i.e s3://mybucket/myproject_abc/

Now I configured all permissions as per documentation
https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/storage-credentials

"Condition": {
"StringLike": {
"s3:prefix": "myproject_abc/*"
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mybucket/myproject_abc/*",
"arn:aws:s3:::mybucket"
],
"Sid": "AllowS3ActionsForProjectABC"
}
0 Kudos

SP_6721
Honored Contributor II

‎05-22-2025 08:58 AM

Hi @arnas 

Yes, you can give Databricks access to just the S3 folder s3://mybucket/myproject_abc/ without exposing the whole bucket.

IAM Policy should include:

  • Bucket level: Allow s3:ListBucket on arn:aws:s3:::mybucket with a condition for the myproject_abc/ prefix.
  • Object level: Allow s3:GetObject, s3:PutObject, and s3:DeleteObject on arn:aws:s3:::mybucket/myproject_abc/*.
0 Kudos

arnas
New Contributor II
In response to SP_6721

‎05-28-2025 06:56 AM

aws cli works to list the folder, and I am able to download files from inside the folder
aws s3 ls
aws s3 cp

here is my policy, anything I am missing ?

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowObjectActionsInCustomerPrefix",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload",
"s3:ListMultipartUploadParts"
],
"Resource": "arn:aws:s3:::mybucket/folder/*"
},
{
"Sid": "AllowListBucketInCustomerPrefix",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads"
],
"Resource": "arn:aws:s3:::mybucket",
"Condition": {
"StringLike": {
"s3:prefix": "folder/*"
}
}
},
{
"Sid": "AllowAssumeRole",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::MYACCOUNTID:role/mycustomer-databricks-access"
}
]
}

Preview file
23 KB
0 Kudos

arnas
New Contributor II

‎05-28-2025 11:33 PM

Thanks, but no thanks, spam resides in JUNK folder

0 Kudos