cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

StructField Metadata Dictionary - What are the possible keys?

chaosBEE
New Contributor II
I have a Delta Live Table which is being deposited to Unity Catalog. In the Python notebook, I am defining the schema with a series of StructFields, for example:
 
StructField(
    "columnName",
    StringType(),
    True,
    metadata = {
        'comment': "This is a comment"
    }
)
 
This works, however now I am trying to add tags, I have tried the following, but it doesn't seem to work:
 
metadata = {
        'comment': "This is a comment"
        'tags': {
            'tag1': 'x',
            'tag2': 'y',
        }
    }
 
Does anyone know to add tags to columns programmatically using the Python API?
 
Also, does anyone know how to add tags to the specific table, I am using 
dlt.create_streaming_table(), but there doesn't seem to be an option here: Delta Live Tables Python language reference | Databricks on AWS
 
Any help would be greatly appreciated!
 
Thanks,
Riqo
 
2 REPLIES 2

Kaniz_Fatma
Community Manager
Community Manager

Hi @chaosBEE,

  1. Adding Tags to Columns Programmatically: To add tags to columns programmatically, you can leverage the @dlt.table decorator and define your schema with metadata.

  2. Adding Tags to the Specific Table: Unfortunately, the dlt.create_streaming_table() function doesn’t directly support adding tags to the table. However, you can create a materialized view or streaming table using the @dlt.table decorator (as shown above) and specify the desired tags in the schema metadata.

For more details, you can refer to the Delta Live Tables Python language reference.

If you have any further questions, feel free to ask! 😊

 

Thank you Kaniz!

A few follow-up questions:

Could you give an example of each?

For 1. you said "define your schema with metadata", how do you this, what is the syntax for defining metadata with tags, like I said before, the following does not work:

StructField(
    "columnName",
    StringType(),
    True,
    metadata = {
        'comment'"This is a comment",
        'tags': {
            'tag1': 'x',
            'tag2': 'y',
        }
    }
)
 
For 2., again, what is the syntax? Could you provide an example? In the link you have shown, there doesn't seem to be a tags option as shown below.
 
@Dlt.table( name="<name>", comment="<comment>", spark_conf={"<key>" : "<value>", "<key>" : "<value>"}, table_properties={"<key>" : "<value>", "<key>" : "<value>"}, path="<storage-location-path>", partition_cols=["<partition-column>", "<partition-column>"], schema="schema-definition", temporary=False)
 
Also, is this available with @Dlt.append_flow too?
 
Thanks
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!