How to get Databricks table usage by Power BI?

[Mado]

Assume that I have a few tables in Databricks and I want to make a report in Power BI that shows the No. of table views for a specific user.

I mean, how many times has a user queried a specific table? This is the information I want to send from Databrocks to Power BI and use it to make a report.

Is there any way that I can send table usage information from Databricks to Power BI?

[Ajay-Pandey]

Hi @Mohammad Saber​ ,

I think this is possible by creating UC in databricks and use user lineage to find the query execution for each user or you can use table lineage to find all the query which is executed over the table.

[Mado]

@Ajay Pandey​ 

Thanks. Could you give me more hints on how to get table lineage and find all queries executed over the table?

[Ajay-Pandey]

Please refer below doc that will help you to enable and use UC in databricks.

Capture and view data lineage with Unity Catalog | Databricks on AWS

[Mado]

Thanks.

If tables are stored in hive_metastore, I think I cannot use Unity Catalog to get table usage.

[Anonymous]

Hi @Mohammad Saber​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[Mado]

The solution I've found so far:

• Get Databricks Audit Log by Azure Log Analytics workspace, and from there, send the logs to Power BI.

However, I didn't get the logs by "Azure Log Analytics workspace" in the first trial.

[Anonymous]

Hi @Mohammad Saber​ 

 I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.

[Mado]

Thanks for your support.

There was an issue with access to the storage account to write the log files and it was solved.