topic Re: Does Databricks Runtime 7.3+ include built-in Hadoop S3 connector configurations? in Data Engineering

Does Databricks Runtime 7.3+ include built-in Hadoop S3 connector configurations?

Divya_Bhadauria — Tue, 11 Nov 2025 13:43:46 GMT

I came across the KB article S3 connection reset error, which mentions not using the following Spark settings for the Hadoop S3 connector for DBR 7.3 and above:

spark.hadoop.fs.s3.impl com.databricks.s3a.S3AFileSystem
spark.hadoop.fs.s3n.impl com.databricks.s3a.S3AFileSystem
spark.hadoop.fs.s3a.impl com.databricks.s3a.S3AFileSystem

My question is: for Databricks Runtime 7.3 and above, are these configurations already built in by default ?

Re: Does Databricks Runtime 7.3+ include built-in Hadoop S3 connector configurations?

hasnat_unifeye — Tue, 11 Nov 2025 15:44:00 GMT

No, you don’t need to set those on DBR 7.3 and above.
From 7.3+ Databricks already uses the newer Hadoop S3A connector by default, so those com.databricks.s3a.S3AFileSystem settings are not part of the default config and shouldn’t be added.

If they are present I would assume it has been added manually, and removing it should cause no issues