topic Re: What are the best practices for python logging function in databricks? in Data Engineering

What are the best practices for python logging function in databricks?

hpant — Mon, 29 Jul 2024 09:52:42 GMT

Re: What are the best practices for python logging function in databricks?

szymon_dybczak — Mon, 29 Jul 2024 10:33:34 GMT

I think they are really similiar to overall best practices when in comes to python logging, like having centralize logging configuration, using correct log levels etc.
Look for example on below article:

10 Best Practices for Logging in Python | Better Stack Community

Re: What are the best practices for python logging function in databricks?

hpant — Mon, 29 Jul 2024 15:25:25 GMT

Hi @szymon_dybczak,

Thanks for your reply. I wanted to ask if databricks has something similar to Azure Synapse to store the logs, e.g. log Analytics?

Re: What are the best practices for python logging function in databricks?

szymon_dybczak — Mon, 29 Jul 2024 15:34:07 GMT

Hi @hpant ,

You can configure diagnostic settings and send the logs to log analytics or storage accounts. Refer to below documentation:

- Send Databricks app logs to Azure Monitor - Azure Architecture Center | Microsoft Learn