Send System Table Logs to ADLS

[Amber-26]

Hello,

I am working with Unity Catalogue in Azure Databricks. I have enabled the system schemas for my workspace but unable to figure out a way to send these system table logs to ADLS which I have mounted using Azure Databricks Connector. 

Can someone help me with that? This documentation just talks about enabling and disabling. 

#UnityCatalogue 

[daniel_sahal]

@Amber-26 
You can setup Audit Log Delivery to your ADLS. See docu below.

https://learn.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/audit-log-d...

[Amber-26]

@daniel_sahal Thanks for the response.

Is there any way to send Billable Usage Logs, Pricing Logs and others to ADLS as well? Will be of great help if you could share the documentation for that as well.

[karthik_p]

@Amber-26 please follow below steps in "export Billing data" and create custom queries this is one way. mostly system billing schema that you enabled have fields (sku etc.) should have match with main billing sheet .

looks there is no direct way to store system schema table logs

https://learn.microsoft.com/en-us/azure/azure-arc/data/view-billing-data-in-azure

[Amber-26]

Hello @karthik_p , thanks for your response. Although there is no way to send schemas logs to ADLS. One way I found is the perform an SQL query on the schema to get data and send it to ADLS as explained in this documentation

Let me know if this is the right way. Thanks!

[karthik_p]

@Amber-26 you can try this approach, also if you want to have graphical representation of everything you can use lakehouse monitoring feature, where after enabling these tables, you can consume them into dashboards and run them and analyze them