I am running a pipeline job in Databricks and it failed with the following message:

Run failed with error message  
Could not reach driver of cluster 5824-145411-p65jt7uo.

This message is not very descriptive, and I am not able to identify the root cause.

Could you please advise how I can access more detailed error logs or driver logs in such cases? Is there a way to capture the underlying error message that caused the cluster driver to become unreachable, so that I can investigate further?

Any guidance or best practices would be greatly appreciated.

Thank you!