We want to switch to Dataset, but Dataset also has a problem of slow unit tests.

When we convert RDD to Dataset, the test takes 3-5x longer.

We try to investigate, and we think the Dataset planning is slow because our data job contains more than 100 joins.

I posted here before but it seems there's no solution to this: https://community.databricks.com/s/question/0D53f00001gEjCdCAK/converting-from-rdd-to-dataset-and-un...