What's the best practice on running ANALYZE on Delta Tables for query performance optimization?

aladda
Databricks Employee
Databricks Employee
 

User16826994223
Databricks Employee
Databricks Employee

Nicely Written

aladda
Databricks Employee
Databricks Employee
  • The ANALYZE Command specifically captures statistics which are relevant for the Cost Based Optimizer to make better decisions.
  • The 32 columns of statistics that Delta auto-collects are specifically for data skipping. This is separate from the ANALYZE command
  • The reason docs currently say Do not run on Delta tables’ is because Its best to run Analyze on Delta tables after completion of any data update/delete operation and when the data has changed by around 10%. This gives the CBO the best and most up-to-date statistics to work with
  • General best practices:

View solution in original post

jlickt
New Contributor II

Super write-up; very useful in understanding how the Delta and non-Delta approaches have evolved.