Internal error message: Failed to launch spark container on instance i-xxxxxxxxxxxxx. Exception: Unexpected internal error, please contact Databricks support

[167559]

I have a docker image for debezium in my ECR repo and added IAM roles from Databricks to pull this image for my cluster and seeing this error when cluster is created

[Debayan]

Hi, Could you please retry a few times? This can be an issue with the docker image download. Also, can be a network issue, and also few other possibilities are also there.

[167559]

I have been trying for more than a week now..... where is the logs for these pulls,....error message is not helpful

[akdm]

If I recall correctly, you'll be able to get better information if you enable logs on the cluster.

Your container probably needs more things installed and you should look through https://github.com/databricks/containers for dependencies. You'd probably need databricksruntime/minimal at a minimum. (If you don't already have it)