Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Hi y'all.I'm trying to export metrics and logs to AWS cloudwatch, but while following their tutorial to do so, I ended up facing this error when trying to initialize my cluster with an init script they provided.This is the part where the script fail...
@digui Did you figure out what to do? We're facing the same issue, the script works for the executors.I was thinking on adding an if that checks if there is log4j.properties and modify it only if it exists
I have tried to read data from Databricks using the following java code.String TOKEN = "token...";
String url = "url...";
Properties properties = new Properties();
properties.setProperty("user", "token");
properties.setProperty("PWD", TOKEN);
Con...
@Binesh J - The issue could be due to the data type of the column is not compatible with getString() method in line#17. use getObject() method to retrieve the value as a generic value and then convert to string.
I'm trying to figure out how a custom appender should be configured in a Databricks environment but I cannot figure it out.When cluster is running, in `driver logs`, time is displayed as 'unknown' for my custom log file and when cluster is stopped, c...
We're having the same problem with 11.3 LTS. Are there any updates? We would like to deliver log4j messages from Databricks Notebooks to custom log files and then upload those to S3 or DBFS. Best
Hi All, Hoping you can help. I am looking to set up a custom logging process that captures application ETL logs and Streaming logs I have set up multiple custom logging appenders using the guide here: https://kb.databricks.com/clusters/overwrite-log4...
Hey there @Laurence Wells Hope you are doing great.Does @Kaniz Fatma 's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?Thanks!
Log4J: part 2Apparently log4j 2.15 is still vulnerable:https://www.lunasec.io/docs/blog/log4j-zero-day-update-on-cve-2021-45046/So beter use version 2.16.But as mentioned in several topics: Databricks does not use an impacted version.
Hi Community, We got an email from our IT Team regarding Apache Log4J Vulnerability. Just wanted to understand if our implementation will be affected by this or not. We are using the following library or package in our notebooksimport org.apache.log4...
