- 9589 Views
- 9 replies
- 4 kudos
Resolved! How to get usage statistics from Databricks or SQL Databricks?
Hi, I am looking for a way to get usage statistics from Databricks (Data Science & Engineering and SQL persona). For example: I created a table. I want to know how many times a specific user queried that table.How many times a pipeline was triggered?...
- 9589 Views
- 9 replies
- 4 kudos
- 4 kudos
You can use System Tables, now available in Unity Catalog metastore, to create the views you described. https://docs.databricks.com/en/admin/system-tables/index.html
- 4 kudos
- 2970 Views
- 4 replies
- 4 kudos
Resolved! DLT pipeline run cost
Hi team,I am looking for a way to find DBU cost for DLT clusters, does it get stored anywhere I have been looking into event_logs but did not find information related to cost. it does have cluster resource utilization details.here is what I found, co...
- 2970 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Chhaya Vishwakarma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 4 kudos
- 1923 Views
- 1 replies
- 1 kudos
Resolved! In Python, Streaming read by DLT from Hive Table
I am pulling data from Google BigQuery and writing it to a bronze table on an interval. I do this in a separate continuous job because DLT did not like the BigQuery connector calling collect on a dataframe inside of DLT.In Python, I would like to rea...
- 1923 Views
- 1 replies
- 1 kudos
- 1 kudos
The below code is a solution. I was missing that I could read from a table with `spark.readStream.format("delta").table("...")`. Simple. Just missed it. This is different than `dlt.read_stream()` which appears in the examples a lot.This is referenced...
- 1 kudos