[CHECKPOINT_RDD_BLOCK_ID_NOT_FOUND] Checkpoint block not found!
Either the executor that originally checkpointed this partition is no longer alive, or the original RDD is unpersisted.

After switching from reliable checkpoint() to localCheckpoint() to improve performance, my job started failing intermittently with the following error on Spark 3.5.2. The failures are not consistent sometimes the job completes successfully, and other times it fails at the same stage, which makes it hard to reproduce and debug.

• Would provisioning a larger cluster (more executor memory/disk) keep the local checkpoint data alive longer and avoid these failures?