Set custom log messages in Scala:
%scala
import org.slf4j.{Logger, LoggerFactory}
val customLogs = LoggerFactory.getLogger("CustomLogs")
customLogs.info("Testing INFO Logs")
customLogs.warn("Testing WARN Logs")
customLogs.error("Testing ERROR Logs")
print("This message will be in stdout instead of log4j")
Set custom logs messages in Python
%python
log4jLogger = spark.sparkContext._jvm.org.apache.log4j
customLogs = log4jLogger.LogManager.getLogger("CustomLogs")
customLogs.info("Testing INFO Logs")
customLogs.warn("Testing WARN Logs")
customLogs.error("Testing ERROR Logs")
print("This message will be in stdout instead of log4j")