Format when specifying docker_image url?

mrstevegross
Contributor III

I am providing a custom Docker image to my Databricks/Spark job. I've created the image and uploaded it to our private ECR registry (the URL is `472542229217.dkr.ecr.us-west-2.amazonaws.com/tectonai/mrstevegross-testing:latest`). Based on the docs (https://docs.databricks.com/en/compute/custom-containers.html#launch-your-compute-using-the-api), however, it is unclear to me if my string format is correct, since I'm getting this error at runtime:

 

Cluster '0121-203731-epwa9mar' was terminated. Reason: INVALID_ARGUMENT (CLIENT_ERROR). Parameters: databricks_error_message:Container setup failed because of an invalid request: Exception when verifying docker container image: Image doesn't exist or invalid credential to pull image from 472542229217.dkr.ecr.us-west-2.amazonaws.com/tectonai/mrstevegross-testing:latest. Stdout: Stderr: time="2025-01-21T20:43:05Z" level=fatal msg="Error parsing image name \"docker://472542229217.dkr.ecr.us-west-2.amazonaws.com/tectonai/mrstevegross-testing:latest\": reading manifest latest in 472542229217.dkr.ecr.us-west-2.amazonaws.com/tectonai/mrstevegross-testing: authentication required"

 

Note in particular that the error claims "parsing image name 'docker://..."; I'm wondering whether the "docker://" prefix indicates that I mis-specified the URL. Can anyone advise on how to properly format "472542229217.dkr.ecr.us-west-2.amazonaws.com/tectonai/mrstevegross-testing:latest" for use in the API call?