Felix
Databricks Employee
Databricks Employee

Hi @Md Tahseen Anam​ , have you had a look at https://databrickslabs.github.io/overwatch/gettingstarted/ before? As of now it's only available on AWS and Azure, but I think this is the kind of granularity you're looking for. In general, you need to factor in the costs for DBU and cloud provider compute - unless you want to go really deep and consider secondary costs like storage, v-nets, ... as well. In your specific case I would recommend to give each user their respective cluster (or personal compute to keep costs low) and tag these respectively. You can then aggregate over the tags via the standard cost monitoring solutions on Azure and GCP. Once the serverless offering on the Databricks platform includes workflows and general compute as well, this will provide you with additional insight and opportunity to control cost.