- 5014 Views
- 3 replies
- 0 kudos
calculate median and inter quartile range on spark dataframe
I have a spark dataframe of 5 columns and I want to calculate median and interquartile range on all. I am not able to figure out how do I write udf and call them on columns.
- 5014 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Here is the easiest way to calculate this... https://stackoverflow.com/questions/37032689/scala-first-quartile-third-quartile-and-iqr-from-spark-sqlcontext-dataframe No Hive or windowing necessary.
- 0 kudos