Hi! I currently have this as an old generic template with amends over time to optimize Databricks Spark execution, can you help me to know if this still makes sense for v10-11-12 or if there are new recommendations? Maybe some of this is making my processes slower but I did not having any deprecation warning or suggestion when creating the session.

Thanks!

spark_session = SparkSession.builder \

.config('spark.speculation', 'false') \

.config('sparkyarn.maxAppAttempts', '1') \

.config('spark.databricks.delta.preview.enabled', 'true') \

.config('spark.databricks.delta.merge.joinBasedMerge.enabled', 'true') \

.config('spark.databricks.delta.multiClusterWrites.enabled', 'false') \

.config('spark.databricks.adaptive.autoOptimizeShuffle.enabled', 'true') \

.getOrCreate()