Accessing Application Listening to Port Through Driver Proxy URL

[somedeveloper]

Good afternoon,

I have an application, Evidently, that I am starting a dashboard service for and that listens to an open port. I would like to access this through the driver proxy URL, but when starting the service and accessing it, I am given a 502 Bad Gateway error. This stands in contrast to separate library, Dask, where I am able to start the service and access the dashboard through the proxy URL with no problems. When consulting ChatGPT, it suggested that Dask is able to automatically access its dashboard service through the Proxy URL due to a configuration set up for it due to its popularity, and that the same may be needed for Evidently to be accessible. 

Does anyone know if there is a configuration change that needs to be done to make Evidently accessible through the proxy URL, and if this is truly done out-of-the-box for Dask? 

[VZLA · Accepted solution]

Hello @somedeveloper , thank you for your question about accessing your Evidently dashboard through the driver proxy URL.

The issue likely stems from the lack of automatic integration or configuration for Evidently to work with the Databricks driver proxy, unlike Dask, which benefits from pre-built support in Databricks for exposing its dashboard.

Here are concise questions to guide troubleshooting, along with why they could help:

1. Binding to Host: Is the Evidently service bound to 0.0.0.0 instead of localhost?
   This ensures the service is accessible externally via the proxy.
2. Port Usage: Have you confirmed the port is free and not blocked by other services or firewalls?
   Conflicting ports can prevent the service from being accessible.
3. Custom Proxy Rules: Have you configured any spark.driver.extraJavaOptions or custom reverse proxy rules in the cluster?
   Databricks may not automatically recognize Evidently, so manual proxy configuration might be needed.
4. Logs: Have you reviewed both the service and proxy logs for connection errors or misconfigured headers?
   Logs can reveal why the proxy might be returning a 502 error.

If these checks don’t resolve the issue, it may be necessary to raise a support ticket with Databricks Support directly. Diagnosing such issues often requires detailed triaging from both Spark and platform perspectives, which is best handled through direct interaction with our support team.

[somedeveloper]

The solution was to add --host 0.0.0.0 as an argument to the command. 

[VZLA]

Glad it helped! Thanks for confirming the solution.