nayan_wylde
Esteemed Contributor II

I think you will not be able to set spark configurations in cluster in serverless. But you can put this in notebook.

spark.conf.set(
  "spark.sql.sources.partitionOverwriteMode", "dynamic"
)