how to dynamically perform aggregation on all columns in a data frame even when some columns have different types like int , double string datetime or float in pyspark (i have 140-200 columns and need to perform aggregation/avg on each column)

[STummala]

need to aggregate all the numerical columns but need to this dynamically

[Debayan]

Hi, Have you tried using the aggregate function which may help in this case?

https://docs.databricks.com/sql/language-manual/functions/aggregate.html

[Anonymous]

Hi ​@sandeep tummala​ ,

Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.

Please help us select the best solution by clicking on "Select As Best" if it does.

Your feedback will help us ensure that we are providing the best possible service to you.

Thank you!