Imtiyaz_Shaikh
New Contributor II

The workaround is available here.

Idea is to use spark.conf.set(), spark.conf.get() methods.