cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Can I get cluster logs in an external storage account?

Mado
Valued Contributor II

Hi,

I have two questions about cluster logs.

Q1) It seems that I can only store logs on DBFS. But, is there any way I can send the cluster logs to an Azure storage account?

Note that workspace is not enabled for the Unity Catalog (and is not aimed to be enabled based on my client's request.).

image 

Q2) We can query tables in the notebook running on the cluster (different from the query editor in SQL persona).

I am interested in table usage information (which user queried a specific table at what time).

Do the logs contain table usage information?

1 ACCEPTED SOLUTION

Accepted Solutions

-werners-
Esteemed Contributor III

Q1: if you mount your storage account on databricks, it is available through dbfs (dbfs/mnt/...).

Q2: I think you need to enable the audit logs for that, although I am not sure.

View solution in original post

2 REPLIES 2

-werners-
Esteemed Contributor III

Q1: if you mount your storage account on databricks, it is available through dbfs (dbfs/mnt/...).

Q2: I think you need to enable the audit logs for that, although I am not sure.

Anonymous
Not applicable

Hi @Mohammad Saber​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!