topic How to blacklist an executor on the Spark cluster in Data Engineering https://community.databricks.com/t5/data-engineering/how-to-blacklist-an-executor-on-the-spark-cluster/m-p/18394#M12201 Fri, 25 Jun 2021 22:31:36 GMT brickster_2018 2021-06-25T22:31:36Z How to blacklist an executor on the Spark cluster https://community.databricks.com/t5/data-engineering/how-to-blacklist-an-executor-on-the-spark-cluster/m-p/18394#M12201 Fri, 25 Jun 2021 22:31:36 GMT https://community.databricks.com/t5/data-engineering/how-to-blacklist-an-executor-on-the-spark-cluster/m-p/18394#M12201 brickster_2018 2021-06-25T22:31:36Z Re: How to blacklist an executor on the Spark cluster https://community.databricks.com/t5/data-engineering/how-to-blacklist-an-executor-on-the-spark-cluster/m-p/18395#M12202 <P>This is a lit of configuration keys to enable or alter the blacklist mechanism:</P><P>spark.blacklist.enabled – set to True</P><P>spark.blacklist.task.maxTaskAttemptsPerExecutor (1 by default)</P><P>spark.blacklist.task.maxTaskAttemptsPerNode (2 by default)</P><P>spark.blacklist.application.maxFailedTasksPerExecutor (2 by default)</P><P>spark.blacklist.stage.maxFailedTasksPerExecutor (2 by default)</P><P>spark.blacklist.application.maxFailedExecutorsPerNode (2 by default)</P><P>spark.blacklist.stage.maxFailedExecutorsPerNode (2 by default)</P><P>spark.blacklist.timeout (not set by default)</P><P>spark.blacklist.killBlacklistedExecutors (false by default)</P> Sat, 27 Nov 2021 17:07:53 GMT https://community.databricks.com/t5/data-engineering/how-to-blacklist-an-executor-on-the-spark-cluster/m-p/18395#M12202 Hubert-Dudek 2021-11-27T17:07:53Z