cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Missing warehouse id/metadata for the system compute warehouse table

Danish11052000
New Contributor II

I ran the following queries for a specific warehouse_id = '54a93d2138433216'

 

SELECT * FROM system.billing.usage WHERE usage_metadata.warehouse_id = '54a93d2138433216';
SELECT * FROM system.compute.warehouse_events WHERE warehouse_id = '54a93d2138433216';
SELECT * FROM system.compute.warehouses WHERE warehouse_id = '54a93d2138433216';
 
Observations:
  • Billing usage and warehouse events tables return valid records for this warehouse ID.
    Danish11052000_0-1762164373471.png

     

    Danish11052000_1-1762164374050.png

     

     

 

  • Warehouse table (system.compute.warehouses) returns no row at all, not even with a deleted timestamp.
    Danish11052000_2-1762164373505.png

     

     

Even if the warehouse is deleted, I expected the metadata table to retain a row with a deleted timestamp or status. Is it a known behavior that deleted warehouses are fully purged from system.compute.warehouses? I would appreciate any explanation, reasoning, or confirmation from others who have observed similar behavior.

1 ACCEPTED SOLUTION

Accepted Solutions

MuthuLakshmi
Databricks Employee
Databricks Employee

@Danish11052000 
The issue can happen when the warehouse created is very old (let's say last year), then you may not see the details in system table. 
If they were deleted before the table was created, you'll not see it.

View solution in original post

1 REPLY 1

MuthuLakshmi
Databricks Employee
Databricks Employee

@Danish11052000 
The issue can happen when the warehouse created is very old (let's say last year), then you may not see the details in system table. 
If they were deleted before the table was created, you'll not see it.