Is there a way to make audit on all tables in hive_metastore (no UC), all are external, to check when each has been used for the last time (queried / updated / etc). ?
Apache Ranger or Apache Sentry can be used for auditing Hive activities. If you have set up auditing in one of these tools, you can review the audit logs to see when tables were accessed. Audit logs are typically stored in a separate location, and you'll need to refer to the documentation of the specific tool you are using for more details. You can modify your Hive queries or scripts to log information about table access to a custom log file. This would involve adding logging statements in your Hive scripts or applications.
Thank you for the suggestions, will check both. However for the hive scripts, it's near impossible, as tables are queried ad-hoc from notebooks/ created ad-hoc. but noone is doing any cleanup and I feel audit is very needed ๐
Welcome to Databricks Community: Lets learn, network and celebrate together
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
