column wise sum in PySpark dataframe

[siddhu308]

i have a dataframe of 18000000rows and 1322 column with '0' and '1' value.

want to find how many '1's are in every column ???

below is DataSet

se_00001 se_00007 se_00036 se_00100 se_0010p se_00250

[siddhu308]

seag_00001 seag_00007 seag_00036 seag_00100 seag_0010p seag_00250

1 0 1 0 0 0

[mathan_pillai]

Hi Siddhu,

You can use

df.select(sum("col1"), sum("col2"), sum("col3"))
where col1, col2, col3 are the column names for which you would like to find the sum

please let us know if it answers your question

Thanks