cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

ppang
by New Contributor III
  • 1014 Views
  • 1 replies
  • 0 kudos

Resolved! Job cluster failed to start with custom docker image

Using Databricks Container Services, we have created two custom docker image: one based on nvidia/cuda:11.8.0-runtime-ubuntu22.04 and another based on databricksruntime/standard:12.2-LTS. In either case, we got this error with no specific diagnostics...

  • 1014 Views
  • 1 replies
  • 0 kudos
Latest Reply
ppang
New Contributor III
  • 0 kudos

I managed to get databricksruntime/standard:12.2-LTS to run in Databricks. However, for the CUDA image (nvidia/cuda:11.8.0-runtime-ubuntu22.04), I have only managed to get it to run with Databricks runtime 10.4 LTS. Does anyone know if Databricks run...

  • 0 kudos
Thijs
by New Contributor III
  • 1093 Views
  • 3 replies
  • 4 kudos

How do I define & run jobs that execute scripts that are copied inside a custom DataBricks container?

Hi all, we are building custom Databricks containers (https://docs.databricks.com/clusters/custom-containers.html). During the container build process we install dependencies and also python source code scripts. We now want to run some of these scrip...

  • 1093 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Thijs van den Berg​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...

  • 4 kudos
2 More Replies
tibfab
by New Contributor II
  • 2330 Views
  • 5 replies
  • 0 kudos

How can I build a custom docker image for the ML runtime (e.g. 12.1 ML)?

I successfully built a custom docker image for the Standard runtime following the steps described on the page Customize containers with Databricks Container Services and based on the image databricksruntime/standard:11.3-LTS. However, I cannot find ...

  • 2330 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Tibor Fabian​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!

  • 0 kudos
4 More Replies
Erik_L
by Contributor II
  • 1219 Views
  • 1 replies
  • 1 kudos

Resolved! python 3.10 custom docker image

Goal: To use Python 3.10.4+Why: We have Python repos that are not backward compatible we want to use.What: I have created an image from Databricks example experimental containers already with Ubuntu 22.04 (2 major versions newer than the curre...

  • 1219 Views
  • 1 replies
  • 1 kudos
Latest Reply
Erik_L
Contributor II
  • 1 kudos

After searching for an hour, I realized what I needed to look for. It's the importing the Iterable from collections, which is deprecated in 3.10. I guess Databricks hasn't migrated code, yet. In which case, I'm at a cross-roads. Databricks 3.9, local...

  • 1 kudos
Labels