https://community.databricks.com/t5/data-engineering/how-do-i-choose-which-column-to-partition-by/m-p/20314#M13700 <P>The important factors deciding partition columns are:</P><UL><LI>Even distribution of data. </LI><LI>Choose the column that is commonly or widely accessed or queried. </LI><LI>Do not create multiple levels of partition, as you can end up with a large number of small files. </LI></UL> Thu, 24 Jun 2021 23:22:00 GMT brickster_2018 2021-06-24T23:22:00Z https://community.databricks.com/t5/data-engineering/how-do-i-choose-which-column-to-partition-by/m-p/20313#M13699 <P>I am in the process of building my data pipeline, but I am unsure of how to choose which fields in my data I should use for partitioning. What should I be considering when choosing a partitioning strategy?</P> Thu, 24 Jun 2021 22:06:12 GMT https://community.databricks.com/t5/data-engineering/how-do-i-choose-which-column-to-partition-by/m-p/20313#M13699 User16826992666 2021-06-24T22:06:12Z https://community.databricks.com/t5/data-engineering/how-do-i-choose-which-column-to-partition-by/m-p/20314#M13700 <P>The important factors deciding partition columns are:</P><UL><LI>Even distribution of data. </LI><LI>Choose the column that is commonly or widely accessed or queried. </LI><LI>Do not create multiple levels of partition, as you can end up with a large number of small files. </LI></UL> Thu, 24 Jun 2021 23:22:00 GMT https://community.databricks.com/t5/data-engineering/how-do-i-choose-which-column-to-partition-by/m-p/20314#M13700 brickster_2018 2021-06-24T23:22:00Z