- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-18-2023 10:41 PM
Hi,
I have a DataFrame and different transformations are applied on the DataFrame. I want to display DataFrame after several transformations to check the results.
However, according to the Reference, every time I try to display results, it runs the execution plan again. A solution has been proposed in the reference by saving the DataFrame and then loading it. However, this solution cannot be applied to the platform I am working on.
Is there any other solution to display results a few times in a notebook without re-executing the logic?
Can I use .cache() for this purpose as below:
- df.cache().count()
- df.display()
And since the name of DataFrame will change in the next lines, I repeat it like below:
- df_new.cache().count()
- df_new.display()