cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

how to dynamically perform aggregation on all columns in a data frame even when some columns have different types like int , double string datetime or float in pyspark (i have 140-200 columns and need to perform aggregation/avg on each column)

STummala
New Contributor

need to aggregate all the numerical columns but need to this dynamically

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, Have you tried using the aggregate function which may help in this case?

https://docs.databricks.com/sql/language-manual/functions/aggregate.html

Anonymous
Not applicable

Hi ​@sandeep tummala​ ,

Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.

Please help us select the best solution by clicking on "Select As Best" if it does.

Your feedback will help us ensure that we are providing the best possible service to you.

Thank you!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.