topic Re: Any Databricks system tables contain info of the saved/pre-defined queries in Administration & Architecture

Any Databricks system tables contain info of the saved/pre-defined queries

BryanC — Wed, 18 Sep 2024 15:12:43 GMT

How can I find the saved/pre-defined queries in Databricks system tables?

system.query.history seems NOT having the info, like query-id or query-name

Re: Any Databricks system tables contain info of the saved/pre-defined queries

tapash-db — Wed, 04 Dec 2024 13:56:22 GMT

Hi Bryan, Databricks system tables do not store saved queries. Query history table captures the query execution details, including:

Statement ID
Execution status
User who ran the query
Statement text (if not encrypted)
Statement type
Execution duration
Resource usage

Re: Any Databricks system tables contain info of the saved/pre-defined queries

BryanC — Wed, 04 Dec 2024 14:34:01 GMT

How can we retrieve the information from the "query history table" and present it in our own dashboard/widget?

Re: Any Databricks system tables contain info of the saved/pre-defined queries

tapash-db — Wed, 04 Dec 2024 15:47:11 GMT

Hi Bryan, Depending on the information you want to retrieve, you can write queries and showcase that in a dashboard. Please see these links for sample query and dashboard.

https://medium.com/dbsql-sme-engineering/the-new-databricks-sql-warehouse-advisor-dashboard-a89771bbee3e

https://community.databricks.com/t5/technical-blog/queries-for-cost-attribution-using-system-tables/ba-p/76558

Re: Any Databricks system tables contain info of the saved/pre-defined queries

BryanC — Wed, 04 Dec 2024 16:39:12 GMT

That did not address the requirement to show the saved/pre-defined queries in our own dashboard, while the saved/pre-defined queries cannot be identified from system.query.history

Re: Any Databricks system tables contain info of the saved/pre-defined queries

tapash-db — Wed, 04 Dec 2024 17:00:11 GMT

Hi Bryan, As I mentioned in the previous thread, query history doesn't store saved query information. There is no out of the box feature which can showcase saved query in the dashboard. For accessing saved query you can follow these methods.

Workspace UI: You can view and manage saved queries by clicking on "Workspace" or "Queries" in the sidebar.
Workspace Browser: Queries can be organized into folders along with other Databricks objects in the workspace browser.
API and Connectors: You can create and manage queries using the REST API, JDBC/ODBC connectors, or partner tools without using the Databricks UI.

Please see this link https://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/