Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-03-2023 07:26 AM
Hello @KARTHICK N ,
The default value for spark.sql.files.maxPartitionBytes is 128 MB. These defaults are in the Apache Spark documentation https://spark.apache.org/docs/latest/sql-performance-tuning.html (unless there might be some overrides).
To check the configurations you can navigate to the Environment tab of the Spark UI and check for the config.
Hope that helps.
Thanks & Regards,
Nandini