cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

youngchef
by New Contributor
  • 898 Views
  • 3 replies
  • 3 kudos

Resolved! AWS Instance Profiles and DLT Pipelines

Hey everyone! I'm building a DLT pipeline that reads files from S3 (or tries to) and then writes them into different directories in my s3 bucket. The problem is I usually access S3 with an instance profile attached to a cluster, but DLT does not give...

  • 898 Views
  • 3 replies
  • 3 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

{ "clusters": [ { "label": "default", "aws_attributes": { "instance_profile_arn": "arn:aws:..." } }, { "label": "maintenance", "aws_attributes": { "instance_profile_arn": "arn:aws:..." ...

  • 3 kudos
2 More Replies
Digan_Parikh
by Valued Contributor
  • 688 Views
  • 1 replies
  • 0 kudos

Resolved! S3 bucket mount

If you mount an S3 bucket using an AWS instance profile, does that mounted bucket become accessible to just that 1 cluster or to other clusters in that workspace as well?

  • 688 Views
  • 1 replies
  • 0 kudos
Latest Reply
Digan_Parikh
Valued Contributor
  • 0 kudos

Mounts are global to all clusters but as a best practice, you can use IAM roles to prevent access tot he underlying data. To take this one step further, you can use IAM credential passthrough rather than instance profile because instance profile can ...

  • 0 kudos
Labels