- 1763 Views
- 3 replies
- 3 kudos
Resolved! AWS Instance Profiles and DLT Pipelines
Hey everyone! I'm building a DLT pipeline that reads files from S3 (or tries to) and then writes them into different directories in my s3 bucket. The problem is I usually access S3 with an instance profile attached to a cluster, but DLT does not give...
- 1763 Views
- 3 replies
- 3 kudos
- 3 kudos
{ "clusters": [ { "label": "default", "aws_attributes": { "instance_profile_arn": "arn:aws:..." } }, { "label": "maintenance", "aws_attributes": { "instance_profile_arn": "arn:aws:..." ...
- 3 kudos
- 1181 Views
- 1 replies
- 0 kudos
Resolved! S3 bucket mount
If you mount an S3 bucket using an AWS instance profile, does that mounted bucket become accessible to just that 1 cluster or to other clusters in that workspace as well?
- 1181 Views
- 1 replies
- 0 kudos
- 0 kudos
Mounts are global to all clusters but as a best practice, you can use IAM roles to prevent access tot he underlying data. To take this one step further, you can use IAM credential passthrough rather than instance profile because instance profile can ...
- 0 kudos