cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Jaeseon
by • New Contributor II
  • 1113 Views
  • 1 replies
  • 0 kudos

Failed to import `Ray` on jupyter notebook.

While working on my school's Linux server, I encountered an issue while attempting to install and import Ray in my Jupyter Notebook. I successfully installed the package ray==2.4.0, but encountered an error when trying to import it, specifically stat...

Data Engineering
Reply
  • 1113 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Jaeseon Song​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
Fed
by • New Contributor III
  • 4946 Views
  • 4 replies
  • 1 kudos

Resolved! Ray dashboard no longer available

Has anyone else experienced the lack of access to the Ray dashboard since this week? Last week worked fine.rom ray.util.spark import setup_ray_cluster   setup_ray_cluster(...)This used to output an HTML block with a link to the dashboard.I can manual...

Data Engineering
Reply
  • 4946 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
Fed
New Contributor III
  • 1 kudos

The reason for the missing dashboard for me was due ​to not having installed some required dependencies. Shout out to the Ray community for their help.​I've submitted a PR (now merged) to add a warning message​ when such dependencies are missing.​​

  • 1 kudos
3 More Replies
Fed
by • New Contributor III
  • 1607 Views
  • 1 replies
  • 2 kudos

Resolved! Ray as a cluster library instead of notebook-scoped library

This article rightly suggests to install `ray` with `%pip`, although it fails to mention that installing it as a cluster library won't work.The reason, I think, is that `setup_ray_cluster` will use `sys.executable` (ie `/local_disk0/.ephemeral_nfs/en...

Data Engineering
Reply
  • 1607 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Fed
New Contributor III
  • 2 kudos

Ugly, but this seems to work for nowimport sys import os import shutil from ray.util.spark import setup_ray_cluster, shutdown_ray_cluster   shutil.copy( "/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/ray", os.path.dirname(sys.execu...

  • 2 kudos
youssefmrini
by • Databricks Employee
  • 1493 Views
  • 2 replies
  • 2 kudos

Resolved! Can I run Ray applications on Databricks ?

With Databricks Runtime 12.0 and above, you can create a Ray cluster and run Ray applications in Databricks with the Ray on Spark API.Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a ...

Data Engineering
Reply
  • 1493 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Youssef Mrini​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 2 kudos
1 More Replies
Mooune_DBU
by • Valued Contributor
  • 1083 Views
  • 0 replies
  • 0 kudos

Spark vs. Ray?

Ray has been getting a lot of traction lately for shining at distributed compute.What are the primary differences between Spark and Ray?In which areas/applications would each be best ? (i.e. Reinforcement Learning)In which cases would it make sense f...

Data Engineering
Reply
  • 1083 Views
  • 0 replies
  • 0 kudos
Labels