Databricks Community
Data Engineering
Re: Spark reparation vs coalesce
Spark reparation vs coalesce
Srikanth_Gupta_
Databricks Employee
Options
Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content
06-14-2021
04:06 PM
Labels:
Labels:
Repartitioning
sajith_appukutt
Databricks Employee
Options
Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content
06-17-2021
09:43 AM
coalesce
avoids a full shuffle and could be used to decrease the number of partitions
repartition
results in a full shuffle and could be used to increase or decrease the number of partitions
