Spark image failed to download or does not exist
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-05-2022 03:04 AM
Good morning, and thank you for the support
In our scheduled job one cluster failed to start with the following error:
```
Run result unavailable: job failed with error message
Unexpected failure while waiting for the cluster to be ready.Cause Unexpected state for cluster: INVALID_ARGUMENT(CLIENT_ERROR): databricks_error_message: Container setup failed because of an invalid request: Spark image "release__10.4.x-snapshot-scala2.12__databricks-universe__head__dab7230__ee00e81__jenkins__9b44ccb__format-2" failed to download or does not exist.
```
This job and related configuration have worked for the past month, as well as for the runs in the days after we received this error.
In the job several cluster are spun up, and on the day of the failure only one had the error.
Where could we find more information or access some log files?
Is there a way to automatically retry to instantiate the cluster? Is setting the `Task retry policy` sufficient to patch this error, or would a retry simply find the cluster in an error state?
I have looked through https://learn.microsoft.com/en-us/azure/databricks/kb/clusters/termination-reasons but could not find a related issue.
Cheers
- Labels:
-
Cluster Failed To Launch