Anonymous
Not applicable

The problem is that gzip isn't splittable. It's going to be single threaded as it tries to uncompress it. Try the option:

.option(“compression”, “gzip”)