topic Re: How to calculate correlation matrix (with all columns at once) in pyspark dataframe? in Data Engineering https://community.databricks.com/t5/data-engineering/how-to-calculate-correlation-matrix-with-all-columns-at-once-in/m-p/29903#M21594 <P></P> <P>got it use -</P> <P>features = dataset.map(lambda row: row[0:])</P> <P>from pyspark.mllib.stat import Statistics </P> <P>corr_mat=Statistics.corr(features, method="pearson")</P> <P></P> Mon, 28 Dec 2015 10:00:58 GMT washim 2015-12-28T10:00:58Z How to calculate correlation matrix (with all columns at once) in pyspark dataframe? https://community.databricks.com/t5/data-engineering/how-to-calculate-correlation-matrix-with-all-columns-at-once-in/m-p/29902#M21593 Mon, 28 Dec 2015 09:07:31 GMT https://community.databricks.com/t5/data-engineering/how-to-calculate-correlation-matrix-with-all-columns-at-once-in/m-p/29902#M21593 washim 2015-12-28T09:07:31Z Re: How to calculate correlation matrix (with all columns at once) in pyspark dataframe? https://community.databricks.com/t5/data-engineering/how-to-calculate-correlation-matrix-with-all-columns-at-once-in/m-p/29903#M21594 <P></P> <P>got it use -</P> <P>features = dataset.map(lambda row: row[0:])</P> <P>from pyspark.mllib.stat import Statistics </P> <P>corr_mat=Statistics.corr(features, method="pearson")</P> <P></P> Mon, 28 Dec 2015 10:00:58 GMT https://community.databricks.com/t5/data-engineering/how-to-calculate-correlation-matrix-with-all-columns-at-once-in/m-p/29903#M21594 washim 2015-12-28T10:00:58Z