- 2471 Views
- 2 replies
- 2 kudos
Resolved! Running multiple linear regressions in parallel (speeding up for loop)
Hi, I am running several linear regressions on my dataframe, in which I run a regression for every unique value in the column "item" , apply the model to a new dataset (vector_new), and at the end union the results as the loop runs. The problem is th...
- 2471 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
@Marcela Bejarano​ :One approach to speed up the process is to avoid using a loop and instead use Spark's groupBy and map functions. Here is an example:from pyspark.ml import Pipeline from pyspark.ml.feature import VectorAssembler from pyspark.ml.reg...
- 2 kudos