stbjelcevic
Databricks Employee
Databricks Employee
In response to pooja_bhumandla

‎12-08-2025 11:38 AM

Hi @pooja_bhumandla ,

To answer your second set of questions directly:

1) Will running ANALYZE for only col33–col42 leave cols 1–32 unchanged?
Yes. With the table property set to include all 42 columns, you can run a targeted recomputation just for the new 10 columns:

   ANALYZE TABLE your_catalog.your_schema.your_table
   COMPUTE DELTA STATISTICS
   FOR COLUMNS col33, col34, col35, col36, col37, col38, col39, col40, col41, col42;
This recomputes Delta file-skipping stats only for the specified columns. Existing stats on cols 1–32 are left as-is.
 
2) Are stats for old and new columns kept together centrally or separately?
They’re maintained together in the Delta log at the file level (one set of file-level stats per file, including the configured columns). There isn’t a separate store per column. The stats schema in the log enumerates which columns have statistics and stores their minimum, maximum, and null counts for each file.
 
3) Does this avoid redundant recomputation?
Yes. Because you target only col33–col42 with FOR COLUMNS, only those columns are recomputed. Existing stats for cols 1–32 remain untouched, so you avoid recomputing work already done.
 
4) Will future data loads automatically maintain stats for all 42 columns?
Yes. After setting delta.dataSkippingStatsColumns (or raising delta.dataSkippingNumIndexedCols), future writes automatically collect file-skipping stats for the configured columns. Property changes affect future stats collection and don’t retroactively recompute for existing files.