System Tables - Billing schema

[Chinu]

Hi Experts!

We enabled UC and also the system table (Billing) to start monitoring usage and cost. We were able to create a dashboard where we can see the usage and cost for each workspace. The usage table in the billing schema has workspace_id but I'd like to map this workspace_id with the workspace name. However, I can't find the workspace name from any of the tables. 

I know System Tables is still in the preview mode and not all the tables are enabled yet. Is there a plan that the "usage" table might have the workspace name as well soon?  Also, if not, what will be the best way for me to create a table with workspace_id and worspace_name? Assuming that I don't have access to the databricks account level. 

[Kaniz]

Hi @Chinu, Certainly! Let’s address your questions regarding workspace IDs and names in Azure Databricks.

Workspace IDs and Names:

• Each Azure Databricks workspace has a unique numerical workspace ID. Additionally, there is a per-workspace URL assigned to every deployment. This URL follows the format adb-<workspace-id>.<random-number>.azuredatabricks.net, where the workspace ID appears immediately after adb- and before the dot (.).
• For example, if you have a per-workspace URL like 1, the instance name is adb-5555555555555555.19.azuredatabricks.net, and the workspace ID is 5555555555555555.
• It’s essential to avoid using legacy regional URLs, as they may not work for new workspaces and exhibit lower performance. Instead, rely on per-workspace URLs for consistency and reliability.

Mapping Workspace IDs to Names:

• Unfortunately, the “usage” table in the billing schema does not directly provide workspace names.
• However, you can create a mapping between workspace IDs and names by leveraging the following approaches:
  • Within a Notebook:
    • Run the following code in a Python or Scala cell within a notebook:spark.conf.get("spark.databricks.clusterUsageTags.clusterOwnerOrgId")
    • This will return the workspace ID.
  • URL Inspection:
    • After logging into your Databricks workspace, examine the URL displayed in your browser’s address bar.
    • Look for a series of numbers displayed after o=; this is the workspace ID.
  • Azure API:
    • Use the Azure API to obtain the per-workspace URL and extract the workspace ID from it.

Creating a Table with Workspace ID and Name:

• Since you don’t have access to the Databricks account level, consider creating a custom table that maps workspace IDs to names within your workspace.
• You can maintain this table by manually associating workspace IDs with their corresponding names.
• Alternatively, if you have a consistent naming convention for workspaces, you could programmatically generate workspace names based on the workspace IDs.

Remember that workspace IDs are stable and unique, making them reliable for mapping purposes. If you encounter any further challenges, feel free to ask for additional assistance! 😊

[Kaniz]

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 
 

[Kaizen]

@Kaniz Im also not seeing the compute names logged in the system billing tables.

Is this located elsewhere?