I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.
However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been more than 7 days that the jobs are running but there is no data for them in billing usage table
Query I used to get the jobIds:
select distinct usage_metadata['job_id'] from system.billing.usage where usage_metadata['job_id'] is not null
I do not see the jobIds I see in UI in this table. The table is enabled and has data about other entities