topic Re: Docker image fails to initalize - Databricks Container Service in Data Engineering

Docker image fails to initalize - Databricks Container Service

azimbhuiyan — Wed, 29 Jan 2025 16:02:55 GMT

I am trying to use a public docker image hosted in Dockerhub for my job. I have not included public_auth field as per this guide since the image type is public and when the cluster is trying to initialize it fails with the error:

Invalid argument: Cannot launch the cluster because the user specified an invalid argument. Internal error message: Container setup failed because of an invalid request: Exception when verifying docker container image: Image doesn't exist or invalid credential to pull image from openrouteservice/openrouteservice:v9.1.0. Stdout: Stderr: time="2025-01-29T15:55:04Z" level=fatal msg="Error parsing image name \"docker://openrouteservice/openrouteservice:v9.1.0\": pinging container registry registry-1.docker.io: Get \"https://registry-1.docker.io/v2/\": EOF"

Any advice?

Re: Docker image fails to initalize - Databricks Container Service

NandiniN — Sat, 01 Feb 2025 06:06:58 GMT

The doc highlights https://docs.databricks.com/en/compute/custom-containers.html

Push your custom base image to a Docker registry. This process is supported with the following registries:

Docker Hub with no auth or basic auth. So auth is not an issue.

Re: Docker image fails to initalize - Databricks Container Service

AskeM — Mon, 12 May 2025 12:37:05 GMT

Did you figure this issue out?
I am having the exact same problem - I am also having trouble finding thorough documentation/examples on the use of DCS, so if you have stumbled upon any links in that regard, i'd also appreciate it. 🙂