Spark UI can give you access to some of this information, just not in real-time. It's also intended for Spark-specific performance information such as job and task breakdowns.

Ganglia metrics can give you real-time metrics along these lines both in real-time and historically.

In the Clusters page for your particular cluster, select the "Metrics" link and you'll have access to the "Ganglia UI" link (for real-time) and the historical snapshots list.screen-shot-2019-05-30-at-40457-pm.png

You can find out more at the Metrics documentation page:

https://docs.databricks.com/user-guide/clusters/metrics.html