Hi Folks -
We released a new metrics view for databricks jobs in Gradient, which helps track and plot the metrics below over time to help engineers understand what's going on with their jobs over time.
- Job cost (DBU + Cloud fees)
- Job Runtime
- Number of core*hours
- Number of workers
- Input data size
- Spill to disk
- Shuffle read/write
Check out our blog here!