Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-14-2023 03:15 AM
Most of the optimisations can be done while selecting the number of partitions we can to create for data, too many would cause a large shuffle operation on wide dependency operations and too less would cause less parallelisation. To minimise the time taken during shuffle operations , use zordering so that data having high chances of falling under the same aggregation, are located on the same or nearby partitions.