I converted a data job fro RDD to Dataset, and I've found that, in prod, the data job runs faster, which is nice.
But unit test runs 3x slower than before.
My best guess is that Dataset spends time doing a lot of stuffs like encoding, optimizing, querying planning.
I wonder if there's a way to disable most of this in unit test since all of these don't matter in a local test.
Alternatively, is there a way to cache the plan, so it doesn't need to be recomputed on each test?
Thank you!
