Taras's answer is correct. Because spark-avro is based on the RDD APIs, the properties must be set in the hadoopConfiguration options.

Please note these docs for configuration using the RDD API: https://docs.azuredatabricks.net/spark/latest/data-sources/azure/azure-datalake.html#access-azure-da...

In Python, you can use

sc._jsc.hadoopConfiguration().set()