COMPUTE DELTA STATISTICS vs COMPUTE STATISTICS - Data Skipping

Sainath368
Contributor

‎05-27-2025 04:05 AM

Hi all,

I recently altered the data skipping stats columns on my Delta Lake table to optimize data skipping. Now, I’m wondering about the best practice for updating statistics:

  • Is running ANALYZE TABLE <table_name> COMPUTE DELTA STATISTICS sufficient after such changes?

  • Or is it necessary to run a full ANALYZE TABLE <table_name> COMPUTE STATISTICS to fully update statistics, especially for newly incoming data?

I want to understand whether COMPUTE DELTA STATISTICS alone keeps the table statistics fully up to date as new data arrives, or if I should regularly run the full COMPUTE STATISTICS command to ensure accuracy.

Thanks in advance for your guidance!

0 Kudos

Advika
Community Manager
Community Manager

‎05-27-2025 08:18 AM

Hello @Sainath368!

Running ANALYZE TABLE <table_name> COMPUTE DELTA STATISTICS is a good practice after modifying data skipping stats columns on a Delta Lake table. However, this command doesn’t update query optimizer stats. For that, you’ll need to run ANALYZE TABLE <table_name> COMPUTE STATISTICS, which collects table and column-level statistics that help the query optimizer generate efficient query plans.

So, the recommended approach will be:

  • Use COMPUTE DELTA STATISTICS after changing data skipping columns or table properties.
  • Periodically, or after significant data changes, run COMPUTE STATISTICS to maintain accurate optimizer statistics.

If you’re using Unity Catalog-managed tables, consider enabling predictive optimization to automate statistics collection and related maintenance. Just note that this applies only to Unity Catalog-managed tables.

0 Kudos