@smoortema , Spark performance tuning is one of the hardest topics to teach or learn, and it’s even tougher to do justice to in a forum thread. That said, I’m really glad to see you asking the question. Tuning is challenging precisely because there are so many moving pieces, which is why AQE was introduced in the first place — to take a large portion of that burden off your shoulders.

If you want to go deeper, structured training is your best path. Databricks offers courses that walk through tuning concepts step by step, and I’m sure platforms like Udemy have solid options as well. A guided approach will give you the most clarity and confidence as you level up your skills.

Hope this helps, Louis.