strange, works fine here. what version of databricks are you on?

What you could do to identify the issue is to output the query plan (.explain).

And also creating a new df for each transformation could help. Like that you can check step by step where things go wrong.