I have a repo that have python files that use the built in logging module. Additionally in some of the notebooks of the repo I want to use logging.debug()/logging.info() instead of print statements everywhere. However when I use the root logger or create my own logger object it picks up logs from py4j and makes it difficult for me to see/use productively.
Here is a screengrab of an MRE
Here is the code:
import logging
#Using the root logger
logging.basicConfig(level=logging.INFO)
logging.debug("This is a debug message")
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical message")
#Using my own logger object
logger = logging.getLogger('my_logger')
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
logger.addHandler(handler)
logger.debug("This is a debug message")
logger.info("This is an info message")
logger.warning("This is a warning message")
logger.error("This is an error message")
logger.critical("This is a critical message")
This is using a Python notebook with DBR 10.4 in Azure
Does anyone know how to use the logging module in a way that doesn't interfere with loggers running in the background with py4j?
I'd like to be able to use it within the notebook, or within a python file that is being imported and used within a notebook.
Thanks