Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2026 01:08 AM
I need to add PII tags at both the table and column levels for a streaming table created using Spark Declarative Pipelines.
I tried applying Unity Catalog tags with the following code inside the SDP Python pipeline:
spark.sql(f"""
ALTER TABLE {table_name}
SET TAGS ({tags_sql})
""")
However, this fails with the following error:
UNSUPPORTED_SPARK_SQL_COMMAND
'${command}' is not supported in spark.sql("...") API in SDP Python.
Supported command: ${supportedCommands}.
What is the correct way to define or apply PII tags for tables and columns created by Spark Declarative Pipelines?