Prabakar
Databricks Employee
Databricks Employee

Setting it either in the notebook or the cluster, both should work. But the better option would be to go with the cluster's spark config.