topic Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant in Data Engineering

DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

luketl2 — Tue, 20 Jan 2026 18:57:44 GMT

I am attempting to ingest csv files from an S3 bucket with Autoloader. Since the schema of the data is inconsistent (each csv may have different headers), I was hoping to ingest the data as Variant following this: https://docs.databricks.com/aws/en/ingestion/variant#-ingest-data-from-cloud-object-storage-as-variant. However, when trying to create a streaming table, I always get the error in the screenshot despite passing table_properties={"delta.feature.variantType-preview": "supported"} as an argument to the flow creating my table. Does the table_properties argument not work with DLT tables for this particular property?

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

pradeep_singh — Wed, 21 Jan 2026 02:58:06 GMT

The VARIANT type for CSV requires Databricks Runtime 16.4+ . can you check whats the runtime your DLT channel is running on .

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

pradeep_singh — Wed, 21 Jan 2026 02:59:22 GMT

Can you also share the exact code you are running to ingest

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

luketl2 — Wed, 21 Jan 2026 15:29:08 GMT

I am on 16.4.15. Prior to attempting to ingest all as variant from csv, I was attempting to turn the csv cols into a map object and then using to_variant_object() to convert to variant, which also faced the same issue.

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

luketl2 — Wed, 21 Jan 2026 15:50:24 GMT

from pyspark import pipelines as dp

import pyspark.sql.functions as F

from utils import col_naming

# groups to ingest

read_groups = ['a', 'b', 'c']

for group in read_groups:

@dp.table(name=f"`{group}`")

def bronze(

table_properties={"delta.feature.variantType-preview": "supported"}

`):`

df = (

spark.readStream

.format("cloudFiles")

.option("cloudFiles.format", "csv")

.option("cloudFiles.inferColumnTypes", "true")

.option("cloudFiles.includeExistingFiles", "true")

.option("header", "true")

.option("singleVariantColumn", "variant_column")

.load(f"s3://bucket_1/{group}/*/*/*/*/*/detail/*.csv")

.withColumn(

"name",

F.regexp_extract(

F.col("_metadata.file_path"),

f"/{group}/" + r"[^/]+/([^/]+)",

)

.withColumn(

"package",

F.regexp_extract(

F.col("_metadata.file_path"),

f"/{group}/" + r"([^/]+)/",

)

return df

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

pradeep_singh — Thu, 22 Jan 2026 13:39:13 GMT

Can you also check with your account admin if this feature is enabled at the workspace level . Since this is still in public preview they might have to enable it .
https://docs.databricks.com/aws/en/delta/variant

Re: DELTA_FEATURES_REQUIRE_MANUAL_ENABLEMENT DLT Streaming Table as Variant

luketl2 — Thu, 22 Jan 2026 16:16:01 GMT

I think I found the issue... I put the table_properties in the wrong place. It goes in the decorator args not the query_function args. My bad 😕