Get Size of a column in Bytes for a Pyspark Data frame

Anbazhagananbut
New Contributor II

Hello All,

I have a column in a dataframe which i struct type.I want to find the size of the column in bytes.it is getting failed while loading in snowflake.

I could see size functions avialable to get the length.how to calculate the size in bytes for a column in pyspark dataframe.

pyspark.sql.functions.size(col)

Collection function: returns the length of the array or map stored in the column.

Please help me on this case.

Thanks