how to generate log files on specific folders

aupres
New Contributor III

Hello! My environments are like below,

 

OS : Windows 11

Spark : spark-4.0.0-preview2-bin-hadoop3

 

And the configuration of spark files 'spark-defaults.conf' and 'log4j2.properties'

spark-defaults.conf

 

spark.eventLog.enabled             true
spark.eventLog.dir                 file:///C:/spark-4.0.0-preview2-bin-hadoop3/sparkeventlogs
spark.serializer                   org.apache.spark.serializer.KryoSerializer
spark.driver.memory                5g
spark.yarn.am.memory               1g
spark.executor.instances           1

spark.executor.extraJavaOptions        -Dlog4j.configuration=file:///C:/spark-4.0.0-preview2-bin-hadoop3/conf/log4j.properties
spark.driver.extraJavaOptions        -Dlog4j.configuration=file:///C:/spark-4.0.0-preview2-bin-hadoop3/conf/log4j.properties

 

log4j2.properties

 

rootLogger = INFO, console, file
rootLogger.level = info
rootLogger.appenderRef.stdout.ref = console, file
rootLogger.appenderRef.file.ref = FileAppender

appender.file=org.apache.log4j.RollingFileAppender 
appender.file.File=C:/spark-4.0.0-preview2-bin-hadoop3/logs
appender.file.MaxFileSize=10MB 
appender.file.MaxBackupIndex=10 
appender.file.layout=org.apache.log4j.PatternLayout 
appender.file.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

 

But log files are not generated on C:/spark-4.0.0-preview2-bin-hadoop3/logs folder. Any idea?