Raymond_Hu

- - Certifications
- - Learning Paths

- - Community Discussions
- - GenAI Insight Hub
- - Get Started Discussions

- - Get Started Resources
- - Events
- - Product Platform Updates
- - Support FAQs
- - Technical Blog
- - What's New in Databricks
- - Get Started Guides
- - Knowledge Sharing Hub
- - Announcements

- - Technical Councils
- - Private Groups
- - Skills@Scale

- - Databricks Community Champions
- - Khoros Community Forums Support (Not for Databricks Product Questions)
- - Databricks Community Code of Conduct
- - Community Newsletter

New Contributor

since

‎06-26-2023

User Stats

1 Posts
0 Solutions
0 Kudos given
0 Kudos received

ConnectException error

09-16-2019

I'm using PySpark on Databricks and trying to pivot a 27753444 X 3 matrix. If I do it in Spark DataFrame: df = df.groupBy("A").pivot("B").avg("C") it takes forever (after 2 hours and I canceled it). If I convert it to pandas dataframe and then pivo...