JAHNAVI
Databricks Employee
Databricks Employee

@pooja_bhumandla We can increase the number of columns on which statistics are collected using the following property: dataSkippingNumIndexedCols

Please refer to the following document for additional information on this : https://docs.databricks.com/aws/en/delta/data-skipping

However, if we add large number of columns to collect the stats it will add extra complexity at the write time for computing the stats and increases transaction‑log metadata size, which affects planning time and driver memory.

Jahnavi N