topic Re: Apply unitycatalog tags programmatically in Data Engineering

Apply unitycatalog tags programmatically

afisl — Mon, 27 Nov 2023 13:47:39 GMT

Hello,

I'm interested in the "Tags" feature of columns/schemas/tables of the UnityCatalog (described here: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/tags)

I've been able to play with them by hand and would now like to integrate them from a programmatic perspective. I was able to QUERY them programmatically (described here https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/information-schema/column_tags#example) but not SET/UPDATE them programmatically.

I think it should be possible to set them programmatically according to the following paragraph in the documentation:

Currently, Azure Databricks supports tagging SQL commands with catalogs, schemas, tables (views, materialized views, streaming tables), and table columns. Learn more about the Databricks SQL tag commands at SQL language reference.

Source: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/tags#manage-tags-with-sql-commands

I've tried UPDATE statement on the information_schema.column_tags table, but get an spark error message:

SparkUnsupportedOperationException: UPDATE TABLE is not supported temporarily.

Any idea how I can set tags programmatically?

Thanks for your support,

Best

Antoine

Re: Apply unitycatalog tags programmatically

afisl — Tue, 28 Nov 2023 08:37:56 GMT

Hi @Retired_mod ,

thanks for your answer. The documentation you quoted is exactly the one I saw.

But this paragraph is for me unclear:"

Manage Tags with SQL Commands:

This feature is available for Databricks Runtime versions 13.3 and above.
You can use Databricks SQL commands to tag catalogs, schemas, tables (views, materialized views, streaming tables), and table columns.
Learn more about the Databricks SQL tag commands in the SQL language reference."

I cannot find the Databricks SQL tag commands in the SQL language reference. Could you help me there please?

Thanks,

Antoine

Re: Apply unitycatalog tags programmatically

melbourne — Tue, 19 Dec 2023 02:47:02 GMT

Hi @afisl

Use following SQL command to add tags programmatically:

alter table <table_name> set tags ('key1' = 'value1', 'key2' = 'value2')

Re: Apply unitycatalog tags programmatically

cltj — Tue, 30 Jan 2024 15:46:15 GMT

Hi lets explore how AI in a comminication channel like this will expoenentially waste the time users use to craft their post and explain their issue or question.

I do not understand why a simple "I dont know, but I will find out" is out of range these days.

Sometimes we can make just as much or even more impact with simplicity without sending users in a loop or down the wrong rabbit hole, perfectly illustrated here by melbourne.

Re: Apply unitycatalog tags programmatically

databass — Mon, 04 Mar 2024 04:17:35 GMT

just confirming, that as at March 2024 you can use SQL to set/unset tags on:

Tables
Table Columns
Views

But NOT on View Columns

however you CAN do this via the UI.

Re: Apply unitycatalog tags programmatically

Mamzy10 — Tue, 11 Jun 2024 10:56:19 GMT

@afisl did you ever get down to the bottom of this? im trying to automatically set PI tags against column with certain words

Re: Apply unitycatalog tags programmatically

dflores_mn — Tue, 23 Jul 2024 18:35:29 GMT

is there a way to apply column tags programmatically to views?

Re: Apply unitycatalog tags programmatically

Jiri_Koutny — Thu, 19 Sep 2024 15:43:28 GMT

Hi, running ALTER TABLE SET TAGS works on views too!

Re: Apply unitycatalog tags programmatically

Marleen — Sun, 23 Mar 2025 07:01:17 GMT

alter table <table_name> alter column <column_name> set tags ('key1' = 'value1', 'key2' = 'value2')