Hi @Nandhini_Kumar, there's many performance metrics available - it depends on what you're looking to do with this data, and how you look to take action in real time. I would strongly recommend mapping out a user journey so you get only the metrics you need, and don't waste time trying to get these all hourly when they won't be used.
- Cluster metrics - you can export these with cluster logging. It's worth noting that ganglia is deprecated for newer runtimes
- Warehouse metrics - available through the API for query metrics
- System tables - if you're using unity catalog, this will update hourly
- Jobs performance - you can use the Jobs API
API docs can be found here: https://docs.databricks.com/api/workspace/introduction
System tables: https://docs.databricks.com/en/administration-guide/system-tables/index.html
