Databricks Community

DaanRademaker · 07-28-2023

Hi,In our setup we use custom docker images on our job clusters. The setup of these images is in line with the docker images found on the databricks github page. The custom image works fine on runtimes, including 13.0. When using anything higher that...

DaanRademaker · 08-21-2023

Interestingly enough I can use the exact same docker image on 13.3LTS Beta with no issues.

DaanRademaker · 08-21-2023

Interestingly enough I can use the exact same image on the 13.3LTS beta runtime without issues.

DaanRademaker · 08-04-2023

Also, this issue can be linked to the following github issue on the Databricks container images github page.https://github.com/databricks/containers/issues/116

DaanRademaker · 08-04-2023

Thank you for your answer. However, I was under the impression that when using docker images in Databricks, Databricks actually injects the spark and hadoop packages into the image. This is how it seems to have always worked. The docker images are si...

DaanRademaker · 08-01-2023

Hi @Debayan! Thanks for your reply.If I attach the docker image to an interactive cluster and run spark.catalog.listTables("dbname")I get the following error:```java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/AlreadyExistsExcepti...

Databricks Community

User Stats

User Activity

Cannot use docker container with runtime 13.1 and above

Re: Cannot use docker container with runtime 13.1 and above

Re: Cannot use docker container with runtime 13.1 and above

Re: Cannot use docker container with runtime 13.1 and above

Re: Cannot use docker container with runtime 13.1 and above

Re: Cannot use docker container with runtime 13.1 and above