question is- when I go to set spark.driver.maxResultSize = 20g in notebook only , it is not taking while executing the job even when I try to get the spark.driver.maxResultSize value in notebook I am getting 20g.

Still need clarification why does it behave like this?