Ganglia metric are not that much helpful and also with cluster start you lose old data .

Question is how to get live metrics and view historical data .

OMS agent are best in that case. i used in Azure databricks and its wonderful .

should be doable in AWS as well with some modification.