system.billing.usage table missing data for jobs running in my databricks account
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-22-2024 03:37 PM
I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.
However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been more than 7 days that the jobs are running but there is no data for them in billing usage table
Query I used to get the jobIds:
select distinct usage_metadata['job_id'] from system.billing.usage where usage_metadata['job_id'] is not null
I do not see the jobIds I see in UI in this table. The table is enabled and has data about other entities
