Access denied error to S3 bucket while running Kinesis spark streaming.

[Avi711]

I get this below error while trying to simulate kinesis streams as mentioned in Databricks 

documentation at https://docs.databricks.com/getting-started/streaming.html

Error:

java.nio.file.AccessDeniedException:

Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID:

com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied; request: PUT https://..

Does anyone faced this issue. Any pointers will be very helpful.

[Pat]

Hi @Aviraj Madkaikar​ ,

Access Denied; request: PUT... . It looks like you don't have permission to write to the location.

Do you use .checkpoint or write data somewhere?

[Avi711 · Accepted solution]

It was an access issue to S3 bucket, resolved by providing appropriate access. Thanks

[jsuarezg]

If you do 

spark.sparkContext._jsc.hadoopConfiguration().set("fs.s3a.access.key", AWS_ACCESS_KEY_ID) + secret with any other secret that has less access than your default one this sometimes happens, so running those commands but with your normal secret will fix the issue