"AWS S3 resource has been disabled" error on job, not appearing on notebook
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-28-2024 02:09 PM
I am getting an "INTERNAL_ERROR" on a databricks job submitted through the API. Which says:
"Run result unavailable: run failed with error message All access to AWS S3 resource has been disabled"
However, when I click on the notebook created by the job and run this through the same cluster there are no errors and I am able to access the data.
Why would this be the case?Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-19-2024 09:49 PM - edited 05-19-2024 09:51 PM
Hi @Retired_mod , thanks for your help. I will look into these issues.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2024 12:30 AM
Hi @Retired_mod , I have tried you suggestions but no luck.
It works fine when I am running via notebook, but when I put that same notebook into a job run and run that, I get the error:
"Run result unavailable: run failed with error message Cannot access AWS bucket"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2024 12:03 AM
In the s3 logs of the run, I am seeing this:
24/05/29 06:39:30 WARN FileSystem: Failed to initialize fileystem dbfs:///: java.io.FileNotFoundException: Bucket user-workspace-s3-bucket does not exist
24/05/29 06:39:30 ERROR DbfsHadoop3: Failed to close FileSystem
java.lang.NullPointerException
