Can't enable "variantType-preview" using DLTs

[Malthe]

Using create_streaming_table and passing table properties as follows, I get an error running the pipeline for the first time:

> Your table schema requires manually enablement of the following table feature(s): variantType-preview.

I'm using this code:

create_streaming_table(
    name=name,
    schema=schema,
    table_properties={
        "delta.feature.variantType-preview": "supported",
        "delta.parquet.compression.codec": "ZSTD",
        "pipelines.reset.allowed": "false",
    }
)

That's a bit surprising because the table shows that this property was indeed set under the Unity Catalog table entry details. I believe this has been working in the past.

The workaround suggested (to manually enable it) does not work because the table was never refreshed:

> Streaming table [REDACTED] needs to be refreshed to execute QUERY. If the table is created from DBSQL, please run REFRESH STREAMING TABLE. If the table is created by a pipeline in Delta Live Tables, please run a pipeline update. SQLSTATE: 55019

[ManojkMohan]

@Malthe 

Root Cause:

The property delta.feature.variantType-preview = supported tells Delta Lake to use the new Variant type, which is still in preview or limited availability

Why It Worked Previously

Earlier Databricks versions implicitly activated such features at table creation.
Recent updates added stricter activation controls

Solution Thinking:

https://learn.microsoft.com/en-us/azure/databricks/ldp/developer/ldp-python-ref-streaming-table

1. Clean Table Creation in Pipeline Code
If possible, delete the old table so your next pipeline run creates a new streaming table with all properties set and no legacy metadata issues. This works best when you control the pipeline end-to-end.

2. For Existing Tables (Manual Activation)
If your table already exists (created previously without the variantType flag fully activated), enable the feature manually using SQL

Immediately after Run

3. Final Step—Trigger Pipeline Update
If the table is under a Delta Live Tables pipeline, any schema or logic change will also trigger Databricks to re-activate its internal metadata:

Add or remove a comment in the create_streaming_table call, or

Edit and rerun your pipeline job.

This step triggers Databricks to finalize table activation and enable the new feature without manual SQL.​

https://docs.databricks.com/aws/en/delta/variant

Be sure to use Databricks Runtime 15.3 or higher for full variant column support. If using DLT (Delta Live Tables), let the pipeline do the activation by deleting and recreating the table within the managed workflow.

[Malthe]

Unfortunately this is on a brand new table.

I was able to track down the runtime that successfully created and refreshed the table:

• Worked:
  dlt:16.4.10-delta-pipelines-dlt-release-dp-20251002-rc0-commit-6bcc9de-image-98a6a38
• Did not work:
  dlt:16.4.10-delta-pipelines-dlt-release-dp-20251009-rc0-commit-8c6b818-image-a20f551

Notice the October 2nd vs 9th release.

This could theoretically be an explanation, that we're seeing a regression here.

The problem persists whether I'm using the CURRENT or PREVIEW channel.

[Malthe]

Here's a self-contained example in SQL that demonstrates the issue:

CREATE TEMPORARY VIEW dummy_kafka AS
SELECT
    'id' AS id,
	'topic' AS topic,
	current_timestamp() AS timestamp,
	1::INTEGER AS partition,
	1::long AS offset,
	parse_json('{}') AS body;

CREATE STREAMING TABLE raw_telemetry (
	id STRING PRIMARY KEY RELY,
	topic STRING NOT NULL,
	timestamp TIMESTAMP NOT NULL,
	partition INTEGER NOT NULL,
	offset LONG NOT NULL,
	body VARIANT NOT NULL
)
TBLPROPERTIES (
    'delta.feature.variantType-preview' = 'supported'
)
AS
SELECT * FROM dummy_kafka;

[Malthe · Accepted solution]

There's a workaround available in most situations which is to first create the table without the VARIANT column, run the pipeline at least once, and then add the column in a subsequent refresh.