I'm using the latest version of Ray (2.3.0) and I don't think it's a Ray issue because it worked perfectly fine until last week. It literally just happened on Monday, and then again everyday over the last 4 days.

I did open an issue a few days ago on GitHub

https://github.com/ray-project/ray/issues/33521

I also don't think it's a Ray issue because if I start ray (ray start --head) on my local machine I can access the dashboard via 127.0.0.1:8265

This is a minimal example of starting a ray cluster similar to based on what ray.util.spark.setup_ray_cluster seems to do behind the scenes

from ray.util.spark.utils import get_random_unused_port
import socket
 
ray_head_ip = socket.gethostbyname(spark.conf.get("spark.driver.host"))
ray_head_port = get_random_unused_port(ray_head_ip, min_port=9000, max_port=10000)
ray_dashboard_port = get_random_unused_port(
    ray_head_ip, min_port=9000, max_port=10000, exclude_list=[ray_head_port]
)
ray_dashboard_agent_port = get_random_unused_port(
    ray_head_ip,
    min_port=9000,
    max_port=10000,
    exclude_list=[ray_head_port, ray_dashboard_port],
)
dashboard_options = [
    "--dashboard-host=0.0.0.0",
    f"--dashboard-port={ray_dashboard_port}",
    f"--dashboard-agent-listen-port={ray_dashboard_agent_port}",
]
 
print("ray start --head" + f" --node-ip-address={ray_head_ip} --port={ray_head_port} " + " ".join(dashboard_options) + " --verbose")

If I then run the command with

%sh
 
ray start --head --node-ip-address=192.168.185.148 --port=9921 --dashboard-host=0.0.0.0 --dashboard-port=9686 --dashboard-agent-listen-port=9032 --verbose

I get the usual output (and no errors about the dashboard, not even in the driver logs) that the cluster has been started and I can confirm that the Ray head node port 9921 is listening

%sh
 
netstat -tulpn | grep :9921

But if I do the same for the dashboard port I don't get anything.

%sh
 
netstat -tulpn | grep :9686

The dashboard port is not listening.