Root Cause:
The property delta.feature.variantType-preview = supported tells Delta Lake to use the new Variant type, which is still in preview or limited availability
Why It Worked Previously
Earlier Databricks versions implicitly activated such features at table creation.
Recent updates added stricter activation controls
Solution Thinking:
https://learn.microsoft.com/en-us/azure/databricks/ldp/developer/ldp-python-ref-streaming-table
1. Clean Table Creation in Pipeline Code
If possible, delete the old table so your next pipeline run creates a new streaming table with all properties set and no legacy metadata issues. This works best when you control the pipeline end-to-end.
2. For Existing Tables (Manual Activation)
If your table already exists (created previously without the variantType flag fully activated), enable the feature manually using SQL
Immediately after Run
3. Final Step—Trigger Pipeline Update
If the table is under a Delta Live Tables pipeline, any schema or logic change will also trigger Databricks to re-activate its internal metadata:
Add or remove a comment in the create_streaming_table call, or
Edit and rerun your pipeline job.
This step triggers Databricks to finalize table activation and enable the new feature without manual SQL.
https://docs.databricks.com/aws/en/delta/variant
Be sure to use Databricks Runtime 15.3 or higher for full variant column support. If using DLT (Delta Live Tables), let the pipeline do the activation by deleting and recreating the table within the managed workflow.
