fwiw: you do not have to be a scala wiz to work on spark in scala.

Older spark articles are often about scala in spark (before python took over).

You will notice it is a lot like pyspark, but way way better.  typing, immutability, things like leftfold and typedLit etc.
Sad scala did not get traction...