Brahmareddy
Esteemed Contributor II

You’re right @KUMAR__111—tracking costs for jobs on ALL_PURPOSE_COMPUTE clusters can be tricky since DBU usage isn’t directly tied to specific jobs. When multiple jobs run in parallel on the same cluster, it’s challenging to allocate costs accurately. Consider using cluster tags to label clusters by job, which can help with grouping costs even when jobs share clusters. Running job-specific clusters for key workloads could provide clearer cost attribution. You could also cross-reference job logs with cluster usage metrics, though this can be manual. Leveraging the Databricks REST API can help gather more detailed metrics to better estimate costs per job.

Just a thought. Give a try and let me know.

Regards,

Brahma