Databricks

Fed · 03-22-2023

Has anyone else experienced the lack of access to the Ray dashboard since this week? Last week worked fine.rom ray.util.spark import setup_ray_cluster setup_ray_cluster(...)This used to output an HTML block with a link to the dashboard.I can manual...

Fed · 03-10-2023

This article rightly suggests to install `ray` with `%pip`, although it fails to mention that installing it as a cluster library won't work.The reason, I think, is that `setup_ray_cluster` will use `sys.executable` (ie `/local_disk0/.ephemeral_nfs/en...

Fed · 01-26-2023

Tree-based estimators in pyspark.ml have an argument called checkpointIntervalcheckpointInterval = Param(parent='undefined', name='checkpointInterval', doc='set checkpoint interval (>= 1) or disable checkpoint (-1). E.g. 10 means that the cache will ...

Fed · 04-11-2023

The reason for the missing dashboard for me was due to not having installed some required dependencies. Shout out to the Ray community for their help.I've submitted a PR (now merged) to add a warning message when such dependencies are missing.

Fed · 03-23-2023

I'm using the latest version of Ray (2.3.0) and I don't think it's a Ray issue because it worked perfectly fine until last week. It literally just happened on Monday, and then again everyday over the last 4 days.I did open an issue a few days ago on ...

Fed · 03-10-2023

Ugly, but this seems to work for nowimport sys import os import shutil from ray.util.spark import setup_ray_cluster, shutdown_ray_cluster shutil.copy( "/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/ray", os.path.dirname(sys.execu...

Fed · 01-06-2023

I've encountered the same warning when running this notebook from DA.https://github.com/databricks-academy/scalable-machine-learning-with-apache-spark-english/blob/published/ML%2002%20-%20Linear%20Regression%20I.pyI've managed to get rid of that war...

Databricks

User Stats

User Activity

Ray dashboard no longer available

Ray as a cluster library instead of notebook-scoped library

Setting checkpoint directory for checkpointInterval argument of estimators in pyspark.ml

Re: Ray dashboard no longer available

Re: Ray dashboard no longer available

Re: Ray as a cluster library instead of notebook-scoped library

Re: How to fix "WARNING mlflow.utils.environment" when run mlflow in Databricks?