Does Databricks Runtime 7.3+ include built-in Hadoop S3 connector configurations?

Divya_Bhadauria
New Contributor III

‎11-11-2025 05:43 AM

I came across the KB article S3 connection reset error, which mentions not using the following Spark settings for the Hadoop S3 connector for DBR 7.3 and above:

spark.hadoop.fs.s3.impl com.databricks.s3a.S3AFileSystem
spark.hadoop.fs.s3n.impl com.databricks.s3a.S3AFileSystem
spark.hadoop.fs.s3a.impl com.databricks.s3a.S3AFileSystem

My question is: for Databricks Runtime 7.3 and above, are these configurations already built in by default ?

 

Labels:
0 Kudos

hasnat_unifeye
Databricks Partner

‎11-11-2025 07:44 AM

No, you don’t need to set those on DBR 7.3 and above.
From 7.3+ Databricks already uses the newer Hadoop S3A connector by default, so those com.databricks.s3a.S3AFileSystem settings are not part of the default config and shouldn’t be added.

If they are present I would assume it has been added manually, and removing it should cause no issues 

0 Kudos