Databricks Community

aranjan99

I am using the system.access.table_lineage table to figure out the tables accessed by sql queries and the corresponding SQL queries. However I am noticing this table missing data or values very often.For eg for sql queries executed by our DBT jobs, ...

aranjan99 · 05-22-2024

I have some jobs running on databricks. I can obtain their jobId from the Jobs UI or List Job Runs API.However when trying to get DBU usage for the corresponding jobs from system.billing.usage, I do not see the same job_id in that table. Its been mor...

aranjan99

The jobs are not long running and complete with in 30seconds. I tried restarting the compute but still not seeing any data for these jobs in the tableThis spport ticket link doesnt work for me, how do I file a support ticket?

aranjan99

Yes it is referencing full table name and these are all SQL tables and not query delta files. If I run the exact same query via a Databricks jobs, the entity run ids are populated. But if I run them via DBT jobs the entity run ids are always NULL

aranjan99

@Kaniz_Fatma I have access to the system.access.table_lineage table. I can see some data in there. My questions is specifically asking about in correct data and missing data in this table.

aranjan99 · 05-28-2024

@Kaniz_Fatma can you point me how to reach out to you? As mentioned earlier, my jobs are completed and are within last week but there is no billing data for them. Hoping you can resolve this soon

aranjan99 · 05-23-2024

The jobs are completed but still not available in that table. Can you point me to how to reach out to you directly?

Databricks Community

User Stats

User Activity

system.access.table_lineage table missing data

system.billing.usage table missing data for jobs running in my databricks account

Re: system.billing.usage table missing data for jobs running in my databricks account

Re: system.access.table_lineage table missing data

Re: system.access.table_lineage table missing data

Re: system.billing.usage table missing data for jobs running in my databricks account

Re: system.billing.usage table missing data for jobs running in my databricks account