cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Serhii
by Contributor
  • 1036 Views
  • 3 replies
  • 1 kudos

Could not launch jobs due to node_type_id (instance) unavailability

I am running hourly job on a cluster using p3.2xlarge GPU instance, but sometimes cluster couldn't start due to instance unavailability. I wander is there is any fallback mechanism to, for example, try a different instance type if one is not availabl...

  • 1036 Views
  • 3 replies
  • 1 kudos
Latest Reply
abagshaw
New Contributor III
  • 1 kudos

 (AWS only) For anyone experiencing capacity related cluster launch failures on non-GPU instance types, AWS Fleet instance types are now GA and available for clusters and instance pools. They help improve chance of successful cluster launch by allowi...

  • 1 kudos
2 More Replies
darkraisisi
by New Contributor
  • 477 Views
  • 0 replies
  • 0 kudos

Is there a way to manually update the cuda required file in the db runtime? There are some rather annoying bugs still in TF 2.11 that have been fixed ...

Is there a way to manually update the cuda required file in the db runtime?There are some rather annoying bugs still in TF 2.11 that have been fixed in TF 2.12.Sadly the latest DB runtime 13.1 (beta) only supports the older TF 2.11 even tho 2.12 was ...

  • 477 Views
  • 0 replies
  • 0 kudos
zzy
by New Contributor III
  • 1249 Views
  • 3 replies
  • 2 kudos

Why is pytorch cuda total memory not aligned with the memory size of GPU cluster I created?

No matter GPU cluster of which size I create, cuda total capacity is always ~16 Gb. Does anyone know what is the issue?The code I use to get the total capacity:torch.cuda.get_device_properties(0).total_memory

  • 1249 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Simon Zhang​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...

  • 2 kudos
2 More Replies
jamesw
by New Contributor II
  • 1614 Views
  • 2 replies
  • 1 kudos

Ganglia not working with custom container services

Setup:custom docker container starting from the "databricksruntime/gpu-conda:cuda11" base image layer10.4 LTS (includes Apache Spark 3.2.1, Scala 2.12)multi-node, p3.8xlarge GPU computeWhen I try to view Ganglia metrics I am met with "502 Bad Gatewa...

image.png image
  • 1614 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @James W​ (Customer)​ , We haven’t heard from you since the last response from @Vivian Wilfred​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please do share that with the community as it can be ...

  • 1 kudos
1 More Replies
vishallakha
by New Contributor II
  • 614 Views
  • 1 replies
  • 2 kudos

How to Enable Files in Repos in DBR 7.3 LTS ML ?

we need a custom version of a GPU cluster with following requirements for a certain project:Ubuntu 18.04Cuda 10.1.Tesla T4 GPU.Availability of /Workspace/Repos folder.All of these requirements are available with DBR ML 7.3 LTS. But one critical compo...

  • 614 Views
  • 1 replies
  • 2 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 2 kudos

Hi, To work with non-notebook files in Databricks Repos, you must be running Databricks Runtime 8.4 or above.https://docs.databricks.com/files/workspace.html#configure-support-for-workspace-files

  • 2 kudos
VictorP
by New Contributor
  • 856 Views
  • 2 replies
  • 4 kudos

Resolved! Does databricks run on GPU?

Does databricks run on GPU?

  • 856 Views
  • 2 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @Victor Prosolin​  , We haven’t heard from you on the last response from @Ron DeFreitas​ , and I checked back to see if his response helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.

  • 4 kudos
1 More Replies
Anonymous
by Not applicable
  • 1794 Views
  • 4 replies
  • 2 kudos

Resolved! Anyone using RAPIDS and cuGraph on a current runtime?

We're in the process of migrating a large graph computation workload to nvidia RAPIDS + cuGraph for GPU acceleration. The package isn't a part of the base runtime and it is available by conda package management only, so can't be installed via init sc...

  • 1794 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Thanks @Prabakar Ammeappin​ , we're looking at this. Strangely, the last commit removed the rapids libraries from the base cuda-images. We're adding them back in.

  • 2 kudos
3 More Replies
Labels