Hi @VMeghraj, To increase the number of cores for Spark History Server, you can modify the spark.history.fs.numReplayThreads
Configuration parameter.
You can set the desired number of cores by modifying the value of spark.history.fs.numReplayThreads
in the Spark configuration file (spark-defaults.conf
or spark.conf
).
For example, if you want to increase the number of cores to 50% of the available seats, you can set the value to spark.history.fs.numReplayThreads=50%
.
After modifying the configuration file, you need to restart the Spark History Server for the changes to take effect.