cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How can I build a custom docker image for the ML runtime (e.g. 12.1 ML)?

tibfab
New Contributor II

I successfully built a custom docker image for the Standard runtime following the steps described on the page Customize containers with Databricks Container Services and based on the image databricksruntime/standard:11.3-LTS. However, I cannot find an appropriate base image for an ML runtime.

The only thing I can come up with is to export the list of all Python packages via pip freeze from a notebook in a running ML cluster and build the Docker image based on that list, but that seems tedious.

Also I'm not sure even that could work with the base image for the Standard runtime. There might be other dependencies missing apart from Python packages.

Has anyone an idea how this could be achieved?

5 REPLIES 5

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi,

Could you please confirm which ML runtime version is needed to create custom docker image?

tibfab
New Contributor II

12.1 ML

Anonymous
Not applicable

Hi @Tibor Fabian​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

tibfab
New Contributor II

Hi, no I'm not. There is still no Docker image in Docker Hub to customize runtime ML 12.1 easily.

Anonymous
Not applicable

Hi @Tibor Fabian​ 

Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.