Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-31-2023 06:22 AM
Hi,
I ran into the same problem today. The core of my problem was with an aggregation and join, my stream does not generate massive amounts of data but it still used 200 shuffle partitions. After scaling this down to 2 (you have to clear checkpoints to take effect) my transactions went down significantly. Hope this helps!