cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

System Tables / Audit Logs action_name createWarehouse/createEndpoint

Hertz
New Contributor II

I am creating a cost dashboard across multiple accounts. I am working get sql warehouse names and warehouse ids so I can combine with system.access.billing on warehouse_id.  But the only action_names that include both the warehouse_id and warehouse_name is the editEndpoint/editWarehouse and deleteEnpoint/deleteWarehouse  actions. Is there another was to do this without have to create a bearer token for each account and using the warehouse api? You would think the warehouse_id would also be in the createWarehouse/createEndpoint actions data.

2 REPLIES 2

Kaniz_Fatma
Community Manager
Community Manager

Hi @Hertz, You’re correct that the editEndpoint/editWarehouse and deleteEndpoint/deleteWarehouse actions include both the warehouse_id and warehouse_name. However, the createWarehouse/createEndpoint actions do not include the warehouse_id1.

To get the warehouse names and IDs without creating a bearer token for each account, you could consider the following approach:

  1. Use the Databricks REST API: The Databricks REST API provides an endpoint to get the information for...1. You can use the GET /api/2.0/sql/warehouses/{id} endpoint to get the information for a single SQL warehouse, including its unique identifier (ID) and logical name (name). However, this would still require authentication.

  2. Create a Warehouse Metadata Table: Another workaround is to set up a Databricks job or notebook that periodically queries the system tables to retrieve warehouse information (including names and IDs). You can store this information in a separate table (e.g., a dedicated “Warehouse Metadata” table) wi...2. This approach would not require creating a bearer token for each account.

Please note that these methods still require some form of authentication to access the Databricks API or the Databricks workspace. If you have many accounts, you might want to consider using a centralized identity management system to manage the tokens. This could simplify the process of managing multiple tokens and reduce the overhead of authentication.

I hope this helps! If you have any other questions, feel free to ask. 😊

 

Hertz
New Contributor II

I just wanted to circle back to this. It appears that the ID is returned in the response column of the create action_name.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!