Hubert-Dudek
Databricks MVP

Easiest is to use spark config in advanced options in cluster settings:

spark-config-aws 

more info here https://docs.databricks.com/clusters/configure.html#spark-configuration


My blog: https://databrickster.medium.com/

View solution in original post