Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-16-2025 08:01 PM
The orderBy function in PySpark is expected to perform a global sort, which involves shuffling the data across partitions to ensure that the entire DataFrame is sorted. This is different from sortWithinPartitions, which only sorts data within each partition.
Let me try your program and understand the results further.