Showing results for 
Search instead for 
Did you mean: 
New Contributor
since ‎08-27-2021

User Stats

  • 1 Posts
  • 0 Solutions
  • 0 Kudos given
  • 0 Kudos received

User Activity

I'm using PySpark on Databricks and trying to pivot a 27753444 X 3 matrix. If I do it in Spark DataFrame: df = df.groupBy("A").pivot("B").avg("C") it takes forever (after 2 hours and I canceled it). If I convert it to pandas dataframe and then pivo...