Mado
Valued Contributor II

@-werners- 

Thanks for your help. 

If I assign cached DataFrame into a new variable, which one should I use in the next cells for next transformations. For example, 

  • cached_df = df.cache()
  • cached_df.count()

Then, which of the following options is correct for next transformations:

df2 = (df
  .join(...)
  .select(...)
  .filter(...)         )

or:

df2 = (cached_df
  .join(...)
  .select(...)
  .filter(...)         )