Custom logging using Log4J to a file

Retko — Tue, 29 Aug 2023 13:37:00 GMT

Hello,

I would like to ask for help setting up the log4j.

I want to use log4j (log4j2) to generate custom log messages in my notebook when running.This message would be generated like this: logger.info("some info message") but using log4j not python logging module.

I want to store only this custom messages into a file on the driver for example under /tmp folder and the file name should be as name of the notebook.

It can be similar to this solution Writing PySpark logs in Apache Spark and Databricks | by Ivan Trusov | Medium , but it stores everything under same file and also with many other different log messages.

So is there something I can do to reach this logging requirement?

Thank you.

topic Custom logging using Log4J to a file in Data Engineering

Custom logging using Log4J to a file