NandiniN
Databricks Employee
Databricks Employee

Hello @KARTHICK N​ ,

The default value for spark.sql.files.maxPartitionBytes is 128 MB. These defaults are in the Apache Spark documentation https://spark.apache.org/docs/latest/sql-performance-tuning.html (unless there might be some overrides).

To check the configurations you can navigate to the Environment tab of the Spark UI and check for the config.

Hope that helps.

Thanks & Regards,

Nandini