system.billing.usage table missing data for jobs running in my databricks account

[aranjan99]

I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.
However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been more than 7 days that the jobs are running but there is no data for them in billing usage table

Query I used to get the jobIds:  

select distinct usage_metadata['job_id'] from system.billing.usage where usage_metadata['job_id'] is not null

I do not see the jobIds I see in UI in this table. The table is enabled and has data about other entities

[Kaniz_Fatma]

Hi @aranjan99,

I understand your concern. The system.billing.usage table in Databricks is updated every hour, and it only includes completed t...¹. If your jobs are long-running and have not completed yet, they will not appear in this table¹. Additionally, the table only retains data for the past 30 days¹.

The job_id is unique to each job run, so any changes to the job will result in a new job_id¹. However, the job_id persists across runs, and it’s the run_id that changes¹.

If your jobs are continuously running and have not completed, or if they were failing a few times a ...¹.

You might want to check the status of your jobs and see if they have been completed. If they are still running or failing, their usage data might not appear in the system.billing.usage table. If this doesn’t resolve your issue, I would recommend reaching out to us for further assistance.

[aranjan99]

The jobs are completed but still not available in that table. Can you point me to how to reach out to you directly?

[aranjan99]

@Kaniz_Fatma can you point me how to reach out to you? As mentioned earlier, my jobs are completed and are within last week but there is no billing data for them. Hoping you can resolve this soon

[Kaniz_Fatma]

Hi, @aranjan99. Apologies for the delayed response.

• If you’re not seeing job IDs from the UI or API in the billing table, it’s possible that the job run IDs are not being populated for long-running jobs.
• To address this, consider restarting the compute associated with your jobs to begin recording the jo...^.
• If the issue persists, I recommend reaching out to Databricks support by filing a support ticket for further assistance.

[aranjan99]

The jobs are not long running and complete with in 30seconds. I tried restarting the compute but still not seeing any data for these jobs in the table

This spport ticket link doesnt work for me,  how do I file a support ticket?