Issues when trying to modify log4j.properties
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-15-2022 01:58 PM
Hi y'all.
I'm trying to export metrics and logs to AWS cloudwatch, but while following their tutorial to do so, I ended up facing this error when trying to initialize my cluster with an init script they provided.
This is the part where the script fail
# configure cloudwatch agent for driver & executor
if [ ! -z $DB_IS_DRIVER ] && [ $DB_IS_DRIVER = TRUE ] ; then
sed -i '/^log4j.appender.publicFile.layout/ s/^/#/g' /home/ubuntu/databricks/spark/dbconf/log4j/driver/log4j.properties
sed -i '/log4j.appender.publicFile=com.databricks.logging.RedactionRollingFileAppender/a log4j.appender.publicFile.layout=com.databricks.labs.log.appenders.JsonLayout' /home/ubuntu/databricks/spark/dbconf/log4j/driver/log4j.properties
else
sed -i '/^log4j.appender.console.layout/ s/^/#/g' /home/ubuntu/databricks/spark/dbconf/log4j/executor/log4j.properties
sed -i '/log4j.appender.console.layout=org.apache.log4j.PatternLayout/a log4j.appender.console.layout=com.databricks.labs.log.appenders.JsonLayout' /home/ubuntu/databricks/spark/dbconf/log4j/executor/log4j.properties
fi
This is the error:
sed: can't read /home/ubuntu/databricks/spark/dbconf/log4j/executor/log4j.properties: No such file or directory
I'm not familiar with java at all, so It's being kinda hard to understand what I should do here. Should I install log4j manually or something?
Thanks in advance!
Diego
- Labels:
-
AWS Cloudwatch
-
Init script
-
Log4j
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-05-2022 01:12 AM
Hi @Diego Rodrigues
Does @Kaniz Fatma response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?
We'd love to hear from you.
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-26-2023 04:17 AM
I am facing the same issue . error details
|+ tree /home/ubuntu/databricks/spark/dbconf | |+ cat /home/ubuntu/databricks/spark/dbconf/log4j/driver/log4j2.xml | |+ sed -i '/^log4j.appender.publicFile.layout/ s/^/#/g' /home/ubuntu/databricks/spark/dbconf/log4j/driver/log4j.properties | |sed: can't read /home/ubuntu/databricks/spark/dbconf/log4j/driver/log4j.properties: No such file or directory
Any lead would be appreciated
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-06-2024 08:40 AM
Did you figure out what to do? We're facing the same issue, the script works for the executors.
I was thinking on adding an if that checks if there is log4j.properties and modify it only if it exists