Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-19-2023 12:12 AM
Thanks.
In this reference, it is suggested to save the cached DataFrame into a new variable:
- When you cache a DataFrame create a new variable for it cachedDF = df.cache(). This will allow you to bypass the problems that we were solving in our example, that sometimes it is not clear what is the analyzed plan and what was actually cached. Here whenever you call cachedDF.select(…) it will leverage the cached data.
I didn't understand well the logic behind it.
Do you think it is necessary to save the DataFrame into a new variable in the case that I want to use caching to display the DataFrame?