cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Z-order or Partitioning? Which is better for Data skipping?

User16869510359
Esteemed Contributor

For Delta tables, among Z-order and Partioning which is recommended technique for efficient Data Skipping

1 ACCEPTED SOLUTION

Accepted Solutions

User16869510359
Esteemed Contributor

Partition pruning is the most efficient way to ensure Data skipping. However, choosing the right column for partitioning is very important. It's common to see choosing the wrong column for partitioning can cause a large number of small file problems and in such scenarios, Z-order is the preferred option.

View solution in original post

1 REPLY 1

User16869510359
Esteemed Contributor

Partition pruning is the most efficient way to ensure Data skipping. However, choosing the right column for partitioning is very important. It's common to see choosing the wrong column for partitioning can cause a large number of small file problems and in such scenarios, Z-order is the preferred option.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.