- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-09-2023 07:32 AM
@Mohammad Saber :
It seems that you have correctly configured the Audit logs to be sent to Azure Diagnostic log delivery and you are able to see the table usage information in "DatabricksUnityCatalog" for tables managed by Unity Catalogue. However, you are not able to see any logs related to querying tables or SQL queries.
Regarding the delay in receiving logs, please note that the 15-minute log delivery frequency refers to the frequency at which logs are sent to the log delivery destination (Azure Diagnostic log delivery in your case). However, there can be additional latency in the log processing pipeline, which can cause a delay of up to 24 hours in some cases.
Regarding the missing logs related to table queries or SQL queries, it is possible that these logs are not being captured by the Audit logs. The "getTable" action you see in the logs is related to the creation of the table and not querying it.
To capture the SQL queries, you can enable query logging in Databricks. Once query logging is enabled, you should be able to see SQL queries in the "DatabricksSQL" log table.
To capture the table queries, you can use the Databricks Table Access Control (TAC) feature. This feature allows you to audit and control access to tables in Databricks. You can enable TAC and configure it to audit table access.
Once TAC is enabled, you should be able to see the table access logs in the "DatabricksTableAccessControl" log table. These logs will contain information about the users who accessed the table and the actions they performed (e.g., read, write).
I hope this helps! Let me know if you have any further questions.