"AWS S3 resource has been disabled" error on job, not appearing on notebook

[jvk]

I am getting an "INTERNAL_ERROR" on a databricks job submitted through the API. Which says:

"Run result unavailable: run failed with error message All access to AWS S3 resource has been disabled"

However, when I click on the notebook created by the job and run this through the same cluster there are no errors and I am able to access the data.

Why would this be the case?Thanks

[Kaniz]

Hi @jvk, The “INTERNAL_ERROR” you’re encountering in your Databricks job, along with the message “Run result unavailable: run failed with error message All access to AWS S3 resource has been disabled,” indicates that there’s an issue related to accessing AWS S3 resources. 

1. AWS Credentials Configuration:

   • Ensure that the AWS credentials (access key and secret key) are correctly configured for your Databricks job. If the credentials are missing or incorrect, it can lead to the “access disabled” error.
   • Verify that the same credentials are being used when you run the notebook manually. Sometimes, different credentials might be set in different contexts, causing discrepancies.

2. IAM Roles vs. AWS Keys:

   • Consider using IAM roles instead of AWS keys for accessing S3 resources. IAM roles provide a more secure and manageable way to grant permissions.
   • If you’re switching from AWS keys to IAM roles, unmount any DBFS mount points associated with S3 buckets created using AWS keys. Then, remount them using the IAM role.
   • Avoid using global init scripts to set AWS keys; instead, use cluster-scoped init scripts if necessa...

3. Instance Profiles and IAM Role Passthrough:

   • When using instance profiles (IAM roles), ensure that the correct IAM role is configured. If you’re using IAM Role Passthrough, verify that the meta instance profile is correctly set up².
   • If you’re using a regular instance profile, make sure that IAM Role Passthrough is disabled.
4. Sometimes, issues with the native Parquet reader can cause S3-related errors. You can work around this problem by disabling the native Parquet reader in your Spark configuration:

   spark.conf.set("spark.databricks.io.parquet.nativeReader.enabled", False)
   

5. This might help resolve any compatibility issues with S3 data access.

         Please review your AWS setup, credentials, and configurations.

[jvk]

Hi @Kaniz , thanks for your help. I will look into these issues.