cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Send System Table Logs to ADLS

Amber-26
New Contributor III

Hello,

I am working with Unity Catalogue in Azure Databricks. I have enabled the system schemas for my workspace but unable to figure out a way to send these system table logs to ADLS which I have mounted using Azure Databricks Connector. 

Can someone help me with that? This documentation just talks about enabling and disabling. 

#UnityCatalogue 

5 REPLIES 5

daniel_sahal
Esteemed Contributor

@daniel_sahal Thanks for the response.

Is there any way to send Billable Usage Logs, Pricing Logs and others to ADLS as well? Will be of great help if you could share the documentation for that as well.

karthik_p
Esteemed Contributor

@Amber-26 please follow below steps in "export Billing data" and create custom queries this is one way. mostly system billing schema that you enabled have fields (sku etc.) should have match with main billing sheet .

looks there is no direct way to store system schema table logs

https://learn.microsoft.com/en-us/azure/azure-arc/data/view-billing-data-in-azure

Amber-26
New Contributor III

Hello @karthik_p , thanks for your response. Although there is no way to send schemas logs to ADLS. One way I found is the perform an SQL query on the schema to get data and send it to ADLS as explained in this documentation

Let me know if this is the right way. Thanks!

karthik_p
Esteemed Contributor

@Amber-26 you can try this approach, also if you want to have graphical representation of everything you can use lakehouse monitoring feature, where after enabling these tables, you can consume them into dashboards and run them and analyze them

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group