How can I build a custom docker image for the ML runtime (e.g. 12.1 ML)?

[tibfab]

I successfully built a custom docker image for the Standard runtime following the steps described on the page Customize containers with Databricks Container Services and based on the image databricksruntime/standard:11.3-LTS. However, I cannot find an appropriate base image for an ML runtime.

The only thing I can come up with is to export the list of all Python packages via pip freeze from a notebook in a running ML cluster and build the Docker image based on that list, but that seems tedious.

Also I'm not sure even that could work with the base image for the Standard runtime. There might be other dependencies missing apart from Python packages.

Has anyone an idea how this could be achieved?

[Debayan]

Hi,

Could you please confirm which ML runtime version is needed to create custom docker image?

[tibfab]

12.1 ML

[Anonymous]

Hi @Tibor Fabian​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[tibfab]

Hi, no I'm not. There is still no Docker image in Docker Hub to customize runtime ML 12.1 easily.

[Anonymous]

Hi @Tibor Fabian​ 

Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!