access denied for S3 sources for course materials

[michalklucz]

I've joined the Apache spark for Data Analysts course.

I am in the module 3: https://www.coursera.org/learn/apache-spark-sql-for-data-analysts/lecture/jNNcZ/using-course-materia...

Whenever I am trying to create the structure on my cluster I receive a permission error from AWS side (can't read the source).

an important part of error:

Caused by: java.lang.Throwable: Forbidden; request: HEAD https://databricks-corp-training.s3.us-west-2.amazonaws.com common/dataframes/people-10m.parquet {} Hadoop 3.4.0, aws-sdk-java/1.12.728 Linux/5.15.0-1058-aws OpenJDK_64-Bit_Server_VM/17.0.10+7-Ubuntu-120.04.1 java/17.0.10 scala/2.12.18-bin-db-3-f50da8f kotlin/1.6.21 vendor/Private_Build cfg/retry-mode/legacy cfg/auth-source#unknown com.amazonaws.services.s3.model.GetObjectMetadataRequest; Request ID: W8X1R82BCX0XMZ7D, Extended Request ID: EXSyVCU7t44SDK+clLmWlzB2HtwrXYsi40eETuztKIUxoagtGS7EyXgWMDsoibQhbV9EBrVp2Dc=, Cloud Provider: AWS, Instance ID: i-0af2cf65983b32fb9 credentials-provider: com.amazonaws.auth.BasicAWSCredentials credential-header: AWS4-HMAC-SHA256 Credential=AKIAJBRYNXGHORDHZB4A/20240705/us-west-2/s3/aws4_request signature-present: true (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: W8X1R82BCX0XMZ7D; S3 Extended Request ID: EXSyVCU7t44SDK+clLmWlzB2HtwrXYsi40eETuztKIUxoagtGS7EyXgWMDsoibQhbV9EBrVp2Dc=; Proxy: null)

Coursera support already rejected my case as it's not a 'platform issue' but content.

cluster is ON, classroom has been set up

[Kaniz_Fatma]

Hi @michalklucz, This error typically occurs when the IAM user or role making the request does not have the necessary permissions to access the specified resource.

To resolve this issue, you should check the following:

1. Ensure that the IAM user or role has the s3:GetObject permission on the specific object or the s3:ListBucket permission on the bucket containing the object. Without these permissions, the HEAD operation will fail with a 403 Forbidden error.
2. Verify that the IAM user or role has the correct AWS credentials (access key and secret key) configured in your Databricks cluster. Ensure that the credentials belong to the IAM user or role with the necessary permissions.
3. Check if the S3 bucket has any specific bucket policies or access control lists (ACLs) that might be restricting access to the object. Ensure that the IAM user or role has the necessary permissions granted through these policies.
4. If you are using a cross-account access scenario, make sure that the bucket owner has granted the necessary permissions to the IAM user or role from the other account. This can be done by updating the bucket policy.
5. Ensure that the S3 bucket and object names are correct and match the ones specified in your Databricks code.

[michalklucz]

Hello Fatima!

thank you for your reply. While it's technically correct, this is not applicable here, as we are discussing databricks owned buckets and materials.

here is a link to the course: https://www.coursera.org/learn/apache-spark-sql-for-data-analysts/supplement/P8ag4/course-materials

here is a link to dbc file: https://files.training.databricks.com/courses/moocs/SQLDA/Lessons.dbc

it should be modified on your side, not mine 😞

[Kaniz_Fatma]

Hi @michalklucz, Thank you so much for your response!

To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours).

[nir777]

Hi @michalklucz  I am also facing the same issue. Did you got any solution on it??

[michalklucz]

Hello,thank you for your recommendation

as I can't log in to support (as I only have a community account) I created a ticket: #00499096 later after checking the topic, I also found that I after encountering the error, I instantly created a ticket: #00498644 .

till now, no reply from Friday.

I hope to see any contact soon as I can progress with the course.

[Rishabh_Tiwari]

Hi @michalklucz ,

I am following-up on this discussion. Is the issue stands resolved? Please let us know if you need further assistance.

Thanks

Rishabh

[supplierscave]

Not yet. I am still having the same error.