topic Re: Can I get cluster logs in an external storage account? in Data Governance

Can I get cluster logs in an external storage account?

Mado — Wed, 22 Feb 2023 23:09:00 GMT

Hi,

I have two questions about cluster logs.

Q1) It seems that I can only store logs on DBFS. But, is there any way I can send the cluster logs to an Azure storage account?

Note that workspace is not enabled for the Unity Catalog (and is not aimed to be enabled based on my client's request.).

Q2) We can query tables in the notebook running on the cluster (different from the query editor in SQL persona).

I am interested in table usage information (which user queried a specific table at what time).

Do the logs contain table usage information?

Re: Can I get cluster logs in an external storage account?

-werners- — Thu, 23 Feb 2023 09:33:48 GMT

Q1: if you mount your storage account on databricks, it is available through dbfs (dbfs/mnt/...).

Q2: I think you need to enable the audit logs for that, although I am not sure.

Re: Can I get cluster logs in an external storage account?

Anonymous — Mon, 24 Apr 2023 05:31:30 GMT

Hi @Mohammad Saber

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you.

Cheers!