Databricks Usage Dashboard - Tagging Networking Costs

[4Twannie]

Problem Overview

Our team has successfully integrated Azure Databricks Usage Dashboards to monitor platform-related costs. This addition has delivered valuable insights into our spending patterns. However, we've encountered a tagging issue that's proving to be a blocker.

We've extensively tagged resources across the platform using the tag key "projectRole", aiming to categorize costs effectively. Unfortunately, some costs—especially networking-related ones—remain untagged and appear as <MISMATCHED> when grouped by this tag key. This stops our ability to trace and categorize all costs precisely.

To get an overview of these untagged records, we used the following query:

SELECT *
FROM `system`.billing.usage
WHERE custom_tags.projectRole IS NULL

This query shows all the without any assigned tags. We're now investigating whether it's possible to tag these "hidden" costs?

What We've Tried So Far

We attempted to tag the entire Databricks workspace via the Azure Portal. This approach did manage to tag most of the previously "hidden" costs, including networking ones. However, it introduced a new issue:

• It overwrote existing "projectRole" tags.
• The original "projectRole" tag was reclassified as "x_projectRole".
• In our dashboards, this led to all resources being grouped under the same tag value—disrupting the original tagging structure.

Question

Is there a way to tag these hidden costs—especially networking charges—without disrupting existing tags or causing tag key rewrites? Ideally, we want to keep the integrity of existing "projectRole" values while adding tags to previously untagged items.

Any guidance or best practices would be greatly appreciated!

[mark_ott · Accepted solution]

There is no direct way to tag certain Azure networking resources (such as network interfaces, public IPs, or managed disks) so that their costs inherit custom tags like "projectRole" in cost reports, because many core networking resources either do not support tags or do not propagate custom tags to Azure billing data. This limitation is a well-known challenge in Azure cost allocation and impacts full traceability of costs in Databricks and broader Azure environments.

Workarounds and Best Practices

• Enable Tag Inheritance in Azure Cost Management: Azure Cost Management offers a "tag inheritance" feature that can apply tags from parent billing scopes (such as resource group or subscription) to untagged resources in cost reports. While not modifying the actual Azure resource tags, this setting helps propagate cost allocation logic virtually in billing data, including for some networking charges. This approach does not overwrite or rewrite existing tags but extends the categorization of untagged/mismatched entries in cost reports.

• Azure Policies for Tag Compliance: Use Azure Policy to enforce tagging at deployment. This approach does not retroactively fix already-untaggable resources, but it helps with new deployments. Consider setting a placeholder or default value if a tag is missing to help identify gaps.

• Resource Group and Subscription-Level Cost Grouping: When resource tagging cannot be made granular, leverage grouping by resource group or subscription for allocation of networking and other untagged costs. Keep strict boundaries for projects at the resource group or subscription level to ensure costs aggregate meaningfully even if tags do not apply.

• Manual Cost Attribution: For truly hidden or untaggable network costs, export detailed usage files and allocate costs manually based on known resource relationships or patterns (e.g., cluster/network pairings in Databricks).