cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

danatsafe
by New Contributor
  • 2765 Views
  • 3 replies
  • 0 kudos

Amazon returns a 403 error code when trying to access an S3 Bucket

Hey! So far I have followed along with the Configure S3 access with instance profiles article to grant my cluster access to an S3 bucket. I have also made sure to disable IAM role passthrough on the cluster. Upon querying the bucket through a noteboo...

  • 2765 Views
  • 3 replies
  • 0 kudos
Latest Reply
winojoe
New Contributor III
  • 0 kudos

I had the same issue and I found a solutionFor me, the permission problems only exist when the Cluster's (compute's) Access mode is "Shared No Isolation".  When the Access Mode is either "Shared" or "Single User" then the IAM configuration seems to a...

  • 0 kudos
2 More Replies
gdoron
by New Contributor
  • 847 Views
  • 2 replies
  • 0 kudos

using pyspark can I write to an s3 path I don't have GetObject permission to?

After spark finishes writing the dataframe to S3, it seems like it checks the validity of the files it wrote with: `getFileStatus` that is `HeadObject` behind the scenes.What if I'm only granted write and list objects permissions but not GetObject? I...

  • 847 Views
  • 2 replies
  • 0 kudos
Latest Reply
Lakshay
Esteemed Contributor
  • 0 kudos

It is not possible in my opinion.

  • 0 kudos
1 More Replies
Jyo
by New Contributor II
  • 1283 Views
  • 2 replies
  • 0 kudos

Rollback error - during deploying the workspace through quickstart

These are the steps I followed:1) Under quickstart-> added workspace name-> selected N-virginia (us-east-1), and quick start2) next step: except the password, I haven't edited any of the below:stackname- (default) databricks-workspace-stackparameters...

  • 1283 Views
  • 2 replies
  • 0 kudos
Latest Reply
rams
Contributor
  • 0 kudos

Hi Jyo, I followed the same "quickstart" approach to configure the databricks with aws account and got the same errors. Please can you tell us how to solve the issue if you have already solved. Thank you.

  • 0 kudos
1 More Replies
Juniper_AIML
by New Contributor
  • 1716 Views
  • 2 replies
  • 1 kudos

Resolved! How to setup Instance profile for initializing Databricks Cluster using Docker?

I was trying to start of the Databricks cluster through a docker image. I followed the setup instruction. Excluding the additional setup to setup the IAM role and instance profile as I was facing issues.The image is stored on AWS ECR in a public repo...

Screenshot 2022-02-15 at 2.39.57 PM Screenshot 2022-02-15 at 2.49.03 PM
  • 1716 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Aman Gaurav​ , Please check the below requirements to avail the Databricks Container Services.Note :-Databricks Runtime for Machine Learning and Databricks Runtime for Genomics does not support Databricks Container Services.Databricks Runtime 6.1...

  • 1 kudos
1 More Replies
User16869510359
by Esteemed Contributor
  • 1126 Views
  • 1 replies
  • 0 kudos

Resolved! Getting file permission issues even though I have the right IAM role attached

I am reading data from S3 from a Databricks cluster and the read operation seldom fails with 403 permission errors. Restarting the cluster fixes my issue.

  • 1126 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16869510359
Esteemed Contributor
  • 0 kudos

The main reason for this behavior is : AWS keys are used in addition to the IAM role. Using global init scripts to set the AWS keys can cause this behavior.The IAM role has the required permission to access the S3 data, but AWS keys are set in the Sp...

  • 0 kudos
Labels