Anyone using RAPIDS and cuGraph on a current runtime?

Anonymous
Not applicable

‎11-03-2021 02:51 PM

We're in the process of migrating a large graph computation workload to nvidia RAPIDS + cuGraph for GPU acceleration.

The package isn't a part of the base runtime and it is available by conda package management only, so can't be installed via init script or package list.

We're a bit surprised it isn't part of the base; it's been on cloudera for over a year, for example.

We're going down the custom docker container route and are curious - anyone else doing the same thing? Have you built a dockerfile? If it's useful to others, we'll share our dockerfile and image in the container registry.

Labels:
0 Kudos

Anonymous
Not applicable

‎11-03-2021 03:56 PM

Hi @Brian Gue​! It's nice to meet you and thank you for your question. This sounds interesting and I look forward to seeing what our members come up with. If not, we will see what we can find for you.

0 Kudos

Prabakar
Databricks Employee
Databricks Employee

‎11-16-2021 08:21 AM

Hi @Brian Gue​ Databricks container services for GPU clusters is now in public preview. You can build your docker image to use it on a GPU cluster.

https://docs.databricks.com/clusters/gpu.html#databricks-container-services-on-gpu-clusters

You can find the GPU-supported docker image in Github.

https://github.com/databricks/containers/tree/master/ubuntu/gpu

View solution in original post

0 Kudos

Anonymous
Not applicable

‎11-16-2021 11:32 AM

Thanks @Prabakar Ammeappin​ , we're looking at this. Strangely, the last commit removed the rapids libraries from the base cuda-images. We're adding them back in.

Prabakar
Databricks Employee
Databricks Employee
In response to Anonymous

‎11-17-2021 09:31 AM

Thanks for the update @Brian Gue​ . Hope, adding the rapids libraries to the cuda-image helps to resolve your issue.