I am getting an "INTERNAL_ERROR" on a databricks job submitted through the API. Which says:

"Run result unavailable: run failed with error message All access to AWS S3 resource has been disabled"

However, when I click on the notebook created by the job and run this through the same cluster there are no errors and I am able to access the data.

Why would this be the case?Thanks