Set custom log messages in Scala:

%scala 
 
import org.slf4j.{Logger, LoggerFactory}
 
val customLogs = LoggerFactory.getLogger("CustomLogs")
 
customLogs.info("Testing INFO Logs")
 
customLogs.warn("Testing WARN Logs")
 
customLogs.error("Testing ERROR Logs")
 
print("This message will be in stdout instead of log4j")

Set custom logs messages in Python

%python
 
log4jLogger = spark.sparkContext._jvm.org.apache.log4j 
 
customLogs = log4jLogger.LogManager.getLogger("CustomLogs") 
 
 
customLogs.info("Testing INFO Logs")
 
customLogs.warn("Testing WARN Logs")
 
customLogs.error("Testing ERROR Logs")
 
print("This message will be in stdout instead of log4j")