Setting up custom tags (JobName, JobID, UserId) on an all-purpose cluster

[dmayi]

Hi i want to set up custom tags on an all-purpose cluster for purposes of cost break down and chargebacks.

What: specifically, i want to capture JobName, JobID, UserId who ran job

I can set other custom tags such as Business Unit, Owner... However, the above tags provide more granuallar details.

The real question, since tags are value pairs, what expression should i put in the value for those tags, so that the Databricks engine will populate with values during runtime as it is done for job clusters?

Thank you for your help with guidance.

Regards

--Mayi

[Kaniz · Accepted solution]

Hi @DIEUDONNE MAYI​, The custom cluster tags are only applied to DBU usage reports for clusters launched from pools and do not propagate to cloud resources.

For detailed information about how pool and cluster tag types work together, see Monitor usage using the cluster, pool, and workspace tags.

For convenience, Azure Databricks applies four default tags to each cluster: Vendor, Creator, ClusterName, and ClusterId.

In addition, Azure Databricks applies two default tags on job clusters: RunName and JobId.

On resources used by Databricks SQL, Azure Databricks also applies the default tag. 

SqlWarehouseId.

Warning

Do not assign a custom tag with the key Name to a cluster.

Every cluster has a tag Name Whose value is set by Azure Databricks. If you change the value associated with the key Name, Azure Databricks can no longer track the cluster. Consequently, the cluster might not be terminated after becoming idle and will continue to incur usage costs.

You can add custom tags when you create a cluster. To configure cluster tags:

1. On the cluster configuration page, click the Advanced Options toggle.
2. At the bottom of the page, click the Tags tab.

3.Add a key-value pair for each custom tag. You can add up to 43 custom tags.

For more details, see Monitor usage using cluster, pool, and workspace tags.

[Vidula]

Hey there @DIEUDONNE MAYI​ 

Does @Kaniz Fatma​  response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!