- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-14-2025 04:38 AM
We are running a Deep Clone script to copy Catalogs between Environments; this script is run through a job (run by SP) with DBR 16.4.12.
Some tables are Deep Cloned and other ones are Dropped and Recreated to load partial data. The ones dropped are recreated through a "SHOW CREATE TABLE" result. The issue comes with certain tables with enableRowTracking that it retrieves both 'delta.rowTracking.materializedRowCommitVersionColumnName' and 'delta.rowTracking.materializedRowIdColumnName' which are table properties not allowed in creation, which results in:
[DELTA_UNKNOWN_CONFIGURATION] Unknown configuration was specified: delta.rowTracking.materializedRowCommitVersionColumnName
We have most of the tables with enableRowTracking activated and it does fails only with some of thems. I attach here the CREATE statement of a failing table:
CREATE TABLE Catalog.Schema.Table (
XXX STRING,
XXX STRING,
XXX TIMESTAMP,
XXX STRING,
XXX STRING,
XXX STRING,
XXX INT,
XXX DATE,
XXX STRING,
XXX STRING,
XXX STRING,
XXX STRING,
XXX TIMESTAMP)
USING delta
TBLPROPERTIES (
'delta.columnMapping.mode' = 'name',
'delta.enableChangeDataFeed' = 'true',
'delta.enableDeletionVectors' = 'true',
'delta.enableRowTracking' = 'true',
'delta.feature.appendOnly' = 'supported',
'delta.feature.changeDataFeed' = 'supported',
'delta.feature.columnMapping' = 'supported',
'delta.feature.deletionVectors' = 'supported',
'delta.feature.domainMetadata' = 'supported',
'delta.feature.invariants' = 'supported',
'delta.feature.rowTracking' = 'supported',
'delta.minReaderVersion' = '3',
'delta.minWriterVersion' = '7',
'delta.rowTracking.materializedRowCommitVersionColumnName' = '_row-commit-version-col-UID',
'delta.rowTracking.materializedRowIdColumnName' = '_row-id-col-UID')It's true that we have tested with DBR 17.3 and it works, since it removes those properties; but we are not yet ready to migrate to Scala 2.13 and thus we want to understand if there is some interaction into the Delta Tables features that forces this error and how to avoid.
- Labels:
-
Delta Lake