- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-07-2023 05:01 PM
@Greg Aponte : There is no fastest way to become SPARK expert but would need a lot of dedication and hands on work to get there. I would recommend you to study all the forms of joins - like broadcast join, shuffle hash join, sort merge join. Essentially the number of shuffles need to be as less as possible and to achieve it you should learn the concepts of filtering, re-partition and coalesce. This can come in handy as well. Also please find a lot of youtube summit videos by Bricksters where they explain how to optimize spark codes. Finally, learn and understand what does every execution paramter in spark mean so that you can tweak it to best optmize the code.
Hope this helps! Happy learning! 🙂