โ05-22-2024 03:37 PM
I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.
However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been more than 7 days that the jobs are running but there is no data for them in billing usage table
Query I used to get the jobIds:
โ05-23-2024 07:14 AM
Hi @aranjan99,
I understand your concern. The system.billing.usage
table in Databricks is updated every hour, and it only includes completed t...1. If your jobs are long-running and have not completed yet, they will not appear in this table1. Additionally, the table only retains data for the past 30 days1.
The job_id
is unique to each job run, so any changes to the job will result in a new job_id
1. However, the job_id
persists across runs, and itโs the run_id
that changes1.
If your jobs are continuously running and have not completed, or if they were failing a few times a ...1.
You might want to check the status of your jobs and see if they have been completed. If they are still running or failing, their usage data might not appear in the system.billing.usage
table. If this doesnโt resolve your issue, I would recommend reaching out to us for further assistance.
โ05-23-2024 11:08 AM
The jobs are completed but still not available in that table. Can you point me to how to reach out to you directly?
โ05-28-2024 11:03 AM
@Kaniz_Fatma can you point me how to reach out to you? As mentioned earlier, my jobs are completed and are within last week but there is no billing data for them. Hoping you can resolve this soon
โ06-03-2024 11:33 PM
Hi, @aranjan99. Apologies for the delayed response.
โ06-27-2024 03:06 PM
The jobs are not long running and complete with in 30seconds. I tried restarting the compute but still not seeing any data for these jobs in the table
This spport ticket link doesnt work for me, how do I file a support ticket?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group