koji_kawamura
Databricks Employee
Databricks Employee

Hi @ShivangiB Yes, Liquid cluster is supposed to work that way. The behavior that only applies new keys to the new data supports the characteristic of "Tables with access patterns that change over time". It allows us to change clustering keys based on query pattern changes without rewriting the historical data.

If the historical data needs to use the new clustering keys, we can use OPTIMIZE FULL, which requires I/O costs and time.