Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2021 12:12 PM
This is by design and working as expected. Spark writes the data distributedly.
use of coalesce (1) can help to generate one file, however this solution is not scalable for large data set as it involves bringing the data to one single task.