topic Re: A basic DataFrame transformation query in Data Engineering

A basic DataFrame transformation query

mghildiy — Sun, 14 Aug 2022 05:25:03 GMT

I want to know how dataframe transformations work.

Suppose I have a DataFrame instance df1. I apply some operation on it, say a filter. As every operation gives a new dataframe, so lets say now we have df2. So we have two DataFrame instances now, df1 and df2. Suppose df1 was built for a data whose size is 5mb. After filtering out some data from df1, df2 has, lets say, 3mb size. So does it mean now we have total 8mb of data in dataframes. Is this how things work?

Re: A basic DataFrame transformation query

Vidula — Sun, 11 Sep 2022 07:14:22 GMT

Hi @mghildiy

Does @Kaniz Fatma response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!