Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-18-2015 01:26 PM
This all depends on the dataset size and specific use cases, but, in general, we've seen that Parquet partitions of about 1GB are optimal. For smaller datasets, however, this large partition size may limit parallelism as tasks operate on individual partitions in parallel, so please keep that in mind.