topic What is the difference between a Narrow Transformation and Wide Transformation in Data Engineering https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-narrow-transformation-and-wide/m-p/22117#M15110 Sun, 20 Jun 2021 03:42:15 GMT aladda 2021-06-20T03:42:15Z What is the difference between a Narrow Transformation and Wide Transformation https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-narrow-transformation-and-wide/m-p/22117#M15110 Sun, 20 Jun 2021 03:42:15 GMT https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-narrow-transformation-and-wide/m-p/22117#M15110 aladda 2021-06-20T03:42:15Z Re: What is the difference between a Narrow Transformation and Wide Transformation https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-narrow-transformation-and-wide/m-p/22118#M15111 <UL><LI>Narrow Transformation: In&nbsp;<I>Narrow transformation</I>, all the elements that are required to compute the records in single partition live in the single partition of parent RDD. Ex:- Select, Filter, Union, </LI><LI>Wide Transformation: Wide transformation, all the elements that are required to compute the records in the single partition may live in many partitions of parent RDD. The partition may live in many partitions of parent RDD. Involves a network shuffle and are split between stages. Ex:- GroupBy, Repartition, Sorts</LI></UL><P></P> Sun, 20 Jun 2021 03:44:13 GMT https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-narrow-transformation-and-wide/m-p/22118#M15111 aladda 2021-06-20T03:44:13Z