Hey @Coleman Milliganโ ,
I also faced this type of issue many times
you can add the below configuration in your cluster and it should work.
spark.executor.heartbeatInterval 60s
spark.network.timeout 120s
For more details, you can explore this doc - https://spark.apache.org/docs/latest/configuration.html
you will get all details about the above 2 configuration
If you like my answer, please select it as the best answer and upvote it
Thanks
Aviral Bhardwaj
AviralBhardwaj