Error DELTA_CATALOG_MANAGED_TABLE_UPGRADE_WITH_OTHER_PROPERTIES during catalog commit upgrade

[xwu]

Hi everyone,

I'm encountering an issue with the Catalog Commit functionality while attempting to upgrade a table, following the official Databricks documentation

Catalog commits | Databricks on AWS

When running the following command on a Managed Iceberg Table (contains VARIANT Type data):

ALTER TABLE xxx
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

I get this error:

[DELTA_CATALOG_MANAGED_TABLE_UPGRADE_WITH_OTHER_PROPERTIES] Cannot combine upgrading table xxx to catalog-managed with other property changes. Properties provided: [delta.enableVariantShredding -> true, delta.feature.catalogmanaged -> supported]. SQLSTATE: 429BQ

Even if I set delta.enableVariantShredding = true in a separate command prior to running the upgrade, I still hit the exact same error.

Has anyone run into this issue before, and how did you resolve it?

Thanks in advance!

[Louis_Frolio]

Hi @xwu ,

Good detective work getting this far. Your SQL is valid, and the error message actually tells the story once you unpack it.

Enabling delta.feature.catalogManaged on an existing table is a protocol upgrade, and it has to be a standalone, metadata-only commit. The engine refuses to combine it with any other property change, which is exactly what DELTA_CATALOG_MANAGED_TABLE_UPGRADE_WITH_OTHER_PROPERTIES is telling you. The clue is that delta.enableVariantShredding = true shows up in the error even though you never asked for it in that command. On DBR 17.3 and above, variant shredding is automatically enabled when a table is created with VARIANT columns, so the property is already sitting in your table metadata. The upgrade path appears to carry it into the same commit, which is why setting it in a separate command beforehand doesn't clear the error: that only records the property on the table, it doesn't get it out of the upgrade's way.

Here's what I would try, in order, on a test clone or non-production copy first:

1. Verify the table state and grab diagnostics: DESCRIBE DETAIL xxx; (take a gander at the tableFeatures column), SHOW TBLPROPERTIES xxx;, and DESCRIBE HISTORY xxx;
2. Opt the table out of shredding. Since this is a Managed Delta table, note the property prefix: ALTER TABLE xxx SET TBLPROPERTIES ('delta.enableVariantShredding' = 'false');
3. Run the upgrade by itself: ALTER TABLE xxx SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');
4. If the opt-out alone doesn't clear it, remove the shredding feature entirely before upgrading: ALTER TABLE xxx DROP FEATURE "variantShredding"; Check tableFeatures first for the exact name, since tables enabled during the preview may show variantShredding-preview. Keep in mind this rewrites shredded VARIANT data in place back to the unshredded format, so it isn't free on a large table.
5. Once the catalog upgrade succeeds, re-enable shredding. It only applies to future writes, so run REORG TABLE xxx APPLY (SHRED VARIANT) if you want existing data rewritten too.

Two housekeeping items. Enabling catalog commits on an existing table requires DBR 18.0 or above (16.4 only covers reading, writing, and creating). And do not cancel the upgrade while it's running, since interrupting it can leave the table in a partially upgraded state that locks reads and writes. Rerun the command instead of canceling.

If this sequence still produces the same error on a supported runtime, the public docs don't establish a confirmed workaround, and I would treat it as a product compatibility gap between variant shredding and the catalog commit upgrade on Managed Iceberg. Open a support ticket and include the DESCRIBE DETAIL output, table properties, history, runtime version, and a reproducible test case.

The short version: get the shredding property fully out of the picture, run the upgrade as its own clean commit, then bring shredding back.

References:

• Catalog commits: https://docs.databricks.com/aws/en/tables/features/catalog-commits
• Variant shredding: https://docs.databricks.com/aws/en/tables/features/variant-shredding
• Dropping Delta table features: https://docs.databricks.com/aws/en/delta/drop-feature

Regards, Louis

[amitsharma1707]

This looks like an interaction between the catalog-managed protocol upgrade and VARIANT shredding, rather than an issue with the SQL syntax itself.

delta.feature.catalogManaged needs to be enabled as a standalone table/protocol upgrade. In this case, Databricks also appears to be carrying delta.enableVariantShredding=true into the same metadata change because the table contains VARIANT, which causes:

DELTA_CATALOG_MANAGED_TABLE_UPGRADE_WITH_OTHER_PROPERTIES

I would first check the current state:

DESCRIBE DETAIL catalog.schema.table;
SHOW TBLPROPERTIES catalog.schema.table;
DESCRIBE HISTORY catalog.schema.table;

Then, on a test/non-production table, try temporarily disabling variant shredding:

ALTER TABLE catalog.schema.table
SET TBLPROPERTIES ('delta.enableVariantShredding' = 'false');

Then run the catalog-managed upgrade separately:

ALTER TABLE catalog.schema.table
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

If the error still references enableVariantShredding, it may be necessary to remove the shredding table feature before the upgrade:

ALTER TABLE catalog.schema.table
DROP FEATURE "variantShredding";

Verify the exact feature name in DESCRIBE DETAIL first, since older/preview tables may expose a slightly different feature name.

After the catalog-managed upgrade succeeds, variant shredding can be enabled again:

ALTER TABLE catalog.schema.table
SET TBLPROPERTIES ('delta.enableVariantShredding' = 'true');

One other important check: upgrading an existing table to catalog-managed requires DBR 18.0+.

So I suspect this is currently an edge case in the upgrade path for tables combining Catalog Commits + VARIANT/Variant Shredding, rather than a problem with the documented ALTER TABLE statement itself.

If the same issue persists after removing shredding on DBR 18.0+, I would raise it with Databricks Support with the DESCRIBE DETAIL, table properties, runtime version, and table history.