Anyone using RAPIDS and cuGraph on a current runtime?

[Anonymous]

We're in the process of migrating a large graph computation workload to nvidia RAPIDS + cuGraph for GPU acceleration.

The package isn't a part of the base runtime and it is available by conda package management only, so can't be installed via init script or package list.

We're a bit surprised it isn't part of the base; it's been on cloudera for over a year, for example.

We're going down the custom docker container route and are curious - anyone else doing the same thing? Have you built a dockerfile? If it's useful to others, we'll share our dockerfile and image in the container registry.

[Anonymous]

Hi @Brian Gue​! It's nice to meet you and thank you for your question. This sounds interesting and I look forward to seeing what our members come up with. If not, we will see what we can find for you.

[Prabakar · Accepted solution]

Hi @Brian Gue​ Databricks container services for GPU clusters is now in public preview. You can build your docker image to use it on a GPU cluster.

https://docs.databricks.com/clusters/gpu.html#databricks-container-services-on-gpu-clusters

You can find the GPU-supported docker image in Github.

https://github.com/databricks/containers/tree/master/ubuntu/gpu

[Anonymous]

Thanks @Prabakar Ammeappin​ , we're looking at this. Strangely, the last commit removed the rapids libraries from the base cuda-images. We're adding them back in.

[Prabakar]

Thanks for the update @Brian Gue​ . Hope, adding the rapids libraries to the cuda-image helps to resolve your issue.