topic Databricks runtime from docker hub image in Data Engineering

Databricks runtime from docker hub image

michaelh — Fri, 11 Mar 2022 11:09:23 GMT

Hello,

are databricks runtimes from docker hub ( https://hub.docker.com/r/databricksruntime/standard ) same as actual runtimes inside Databricks? I mean when we made our own docker image from databricksruntime/standard will be there same dependencies, versions etc as in runtime inside databricks? We need to archive older runtimes in bank environment and be able to run old notebooks with exactly same runtime.

And another question about runtime’s version. If I use my docker container in cluster creation with for example runtime version 6.6 but in setup I also select different version in column Databricks runtime version what will happened? Will be column Databricks runtime version ignored?

Re: Databricks runtime from docker hub image

Hubert-Dudek — Fri, 11 Mar 2022 11:22:49 GMT

just build own using in docker FROM databricksruntime/standard:10.x and than it will work correctly with any 10 non beta version (although better test before changing cluster runtime)

Re: Databricks runtime from docker hub image

Atanu — Mon, 21 Mar 2022 19:16:02 GMT

use the 9.x tag for an image that will target a cluster with runtime version Databricks Runtime 9.0. This is for an example. so basically you are running a image based on the version selected.so it better match.

Re: Databricks runtime from docker hub image

jose_gonzalez — Mon, 11 Apr 2022 21:20:35 GMT

Hi @michael henzl ,

Just checking if you still need help with this or not any more? please let us know