Thank you for the answers.

Unfortunately this did not solve the performance issue.

What I did now is I saved the results into a table:

results.write.mode("overwrite").saveAsTable("db.results")

This is probably not the best solution but after I do that I can work with the results data from the table.