Hi @Alex13 , thanks for the detailed minimal repro. Isolating variantType-preview as the trigger was the hard part, and it makes your three questions answerable. Here is my read.

1. Is this expected? Not according to anything public. The only documented statistics limitation is that VARIANT columns themselves do not provide minValues or maxValues. Data skipping is based on per-file statistics, and delta.dataSkippingStatsColumns exists specifically to name the scalar columns that should get them. Nothing in the docs says scalar columns positioned after a VARIANT column lose file-level statistics. Since event_time is explicitly listed and moving it before the VARIANT columns restores pruning, this looks like a preview-feature interaction or regression, not a normal VARIANT limitation. Your own evidence points at the read side rather than stale statistics: if the stats were simply missing from the files, ANALYZE ... COMPUTE DELTA STATISTICS or OPTIMIZE FULL would have fixed it. You can confirm by inspecting the add actions in the _delta_log JSON for the affected files. If minValues and maxValues for event_time are present but the scan still reads every file, the reader is mishandling the stats schema when variantType-preview is on the table. Either way, please open a Databricks Support case with this repro, the exact DBR build, DESCRIBE DETAIL output, table properties, and the query profile. Engineering does not pick up bug reports from Community threads, and this one deserves eyes.

2. The difference between the two features. variantType-preview is the legacy table-feature name from the Public Preview period (DBR 15.3 era). variantType without the suffix is the current stable feature from the open Delta protocol, and the Delta Kernel changelog tracks them as separate reader/writer features. They are not two independent performance modes. A table showing both has either carried forward the older preview metadata or, as in your repro, had the preview property set explicitly on a modern table. Both being present is legal at the protocol level. The practical takeaway from your own experiment: on a current runtime, never set delta.feature.variantType-preview manually. Creating a VARIANT column already enables everything you need.

3. Getting off variantType-preview. I would not keep retrying DROP FEATURE as an in-place fix. Dropping a feature requires removing all traces of it from the data files and transaction log before the protocol can downgrade, the variant features are not in the documented list of removable features, and I could not find a supported command that removes the legacy feature while retaining your VARIANT data. DEEP CLONE will not help either, since clones inherit the source protocol. The practical supported path is the one you found: create a replacement table on DBR 17.3 or above (CTAS, backfill, swap names), which lands you on the current feature model only. There is a silver lining. New tables on 17.3+ get variant shredding automatically, and DBR 18.1+ is recommended for variant statistics collection and data skipping, so the rewrite buys you real performance beyond fixing this issue.

Until the root cause is confirmed, a few things to try on a test copy:

• Keep event_time (and any other skipping or clustering columns) ahead of the VARIANT columns in the schema.
• If a full rewrite is too expensive right now, test ALTER TABLE ... ALTER COLUMN event_time FIRST as a logical reorder. Note this requires column mapping (delta.columnMapping.mode = name), so check that first. Treat it as an experiment, not a documented fix.
• Keep delta.dataSkippingStatsColumns explicitly set to include event_time, recompute statistics after any property or schema change, and confirm the result in the query profile or EXPLAIN.

The fact that behavior changes with the feature flag and with column order is strong evidence of a product issue worth escalating. If you do open a ticket, please circle back here with what support says. Others will hit this.

Public references:

• Variant type support and limitations: https://docs.databricks.com/aws/en/tables/features/variant
• Data skipping and dataSkippingStatsColumns: https://docs.databricks.com/aws/en/tables/data-skipping
• Variant shredding (DBR 17.3+ / 18.1+): https://docs.databricks.com/aws/en/tables/features/variant-shredding
• Drop a Delta Lake table feature: https://docs.databricks.com/aws/en/delta/drop-feature
• ALTER TABLE column positioning: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-alter-table
• Delta Kernel changelog showing both variant features: https://github.com/delta-io/delta-kernel-rs/blob/main/CHANGELOG.md

Regards, Louis