cancel
Showing results for 
Search instead for 
Did you mean: 

Job cluster failed to start with custom docker image

ppang
New Contributor III

Using Databricks Container Services, we have created two custom docker image: one based on nvidia/cuda:11.8.0-runtime-ubuntu22.04 and another based on databricksruntime/standard:12.2-LTS. In either case, we got this error with no specific diagnostics:

Run result unavailable: run failed with error message

Unexpected failure while waiting for the cluster (***-xxxxxx-xxxxxxxx) to be ready: Cluster ***-xxxxxx-xxxxxxxx is in unexpected state Terminated: CONTAINER_LAUNCH_FAILURE(SERVICE_FAULT): instance_id:i-xxxxxxxxxxxxxxxxx,databricks_error_message:Failed to launch spark container on instance xxxxxxxxxxxxxxxxx. Exception: Unexpected internal error, please contact Databricks support.

1 ACCEPTED SOLUTION

Accepted Solutions

ppang
New Contributor III

I managed to get databricksruntime/standard:12.2-LTS to run in Databricks.

However, for the CUDA image (nvidia/cuda:11.8.0-runtime-ubuntu22.04), I have only managed to get it to run with Databricks runtime 10.4 LTS.

Does anyone know if Databricks runtime 11.3 LTS and above supported CUDA images with Databricks container service?

View solution in original post

1 REPLY 1

ppang
New Contributor III

I managed to get databricksruntime/standard:12.2-LTS to run in Databricks.

However, for the CUDA image (nvidia/cuda:11.8.0-runtime-ubuntu22.04), I have only managed to get it to run with Databricks runtime 10.4 LTS.

Does anyone know if Databricks runtime 11.3 LTS and above supported CUDA images with Databricks container service?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.