Hi, your understanding is accurate-with a few nuances about DLT and how it handles full refreshes with external source tables regarding deletes and retention.
During full refresh : Clears the pipeline state and output tables , then reprocesses all d...
Hi, delta.dataSkippingStatsColumns specifies a coma-separated list of column names used by Delta Lake to collect statistics. It will improve the performance by skipping those columns since it will supersede the default behavior of analyzing the first...
Hi @pooja_bhumandla , Yes, deletion vectors are supported for partitioned delta tables in Databricks. It comes as part of storage optimization that allows to delete, update, merge operations to mark existing rows as removed or changed without rewrit...
