cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
sachinmkp1
New Contributor II
since ‎08-27-2021
‎06-26-2023

User Stats

  • 2 Posts
  • 0 Solutions
  • 0 Kudos given
  • 0 Kudos received

User Activity

org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 69 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)

08-23-2021
set spark.conf.set("spark.driver.maxResultSize", "20g") get spark.conf.get("spark.driver.maxResultSize") // 20g which is expected in notebook , I did not do in cluster level setting still getting 4g while executing the spark job , why? because of th...