Redacted check constraint condition in Delta Table

SamAdams — Thu, 10 Oct 2024 22:53:47 GMT

Hello! I have a delta table with a check constraint - it's one of many that a config-driven ETL pipeline of mine generates. When someone edits the config file and deploys the change, I'd like for the check constraint to be updated as well if it's different. The problem I'm running into is that the check constraint's condition is redacted in the table properties, so I can't compare what's in the config with what is in the table properties. I am an admin in this workspace. Is this redacted condition expected, or is it possible to get the actual SQL syntax to show through another setting?

Output from SHOW TBLPROPERTIES or DESCRIBE EXTENDED:

```'delta.minReaderVersion': '1', 'delta.minWriterVersion': '3', 'delta.constraints.sql_data_quality': '*********(redacted)'```

Thanks in advance if anyone knows what's going on.

Re: Redacted check constraint condition in Delta Table

SamAdams — Fri, 11 Oct 2024 01:51:57 GMT

Figured this out with the help of @SamDataWalk 's post https://community.databricks.com/t5/data-engineering/databricks-bug-with-show-tblproperties-redacted-azure-databricks/m-p/93546

It happens because Databricks thinks certain keywords in the constrain query columns like "url" or "userid" are sensitive. You can disable this behavior with

spark.conf.set("spark.databricks.behaviorChange.SC102534CommandRedactProperties.enabled", False)

topic Re: Redacted check constraint condition in Delta Table in Data Engineering

Redacted check constraint condition in Delta Table

Re: Redacted check constraint condition in Delta Table