cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Identity column in DLT using Python

kskistad
New Contributor III

How would I implement the Identity column in Delta Live Tables using Python syntax?

GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( [ START WITH start ] [ INCREMENT BY step ] ) ]

 

1 ACCEPTED SOLUTION

Accepted Solutions

LaurentLeturgez
New Contributor III
New Contributor III

Hi @Kory Skistad​ 

Please find below the table schema definition to use in a python dlt pipeline. You can see it mentions the identity column definition.

@dlt.table(

comment="Raw data on sales",

schema="""

customer_id STRING,

customer_name STRING,

number_of_line_items STRING,

order_datetime STRING,

order_number LONG,

order_day_of_week STRING GENERATED ALWAYS AS (dayofweek(order_datetime))

""",

partition_cols = ["order_day_of_week"])

def sales():

return ("...")

This example has been taken from this page: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-python-ref.html

By the way, be careful about how it is generated regarding you are dealing with a streaming table or not: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-faqs-issues.html#how-do-i-...

HTH

Laurent

View solution in original post

1 REPLY 1

LaurentLeturgez
New Contributor III
New Contributor III

Hi @Kory Skistad​ 

Please find below the table schema definition to use in a python dlt pipeline. You can see it mentions the identity column definition.

@dlt.table(

comment="Raw data on sales",

schema="""

customer_id STRING,

customer_name STRING,

number_of_line_items STRING,

order_datetime STRING,

order_number LONG,

order_day_of_week STRING GENERATED ALWAYS AS (dayofweek(order_datetime))

""",

partition_cols = ["order_day_of_week"])

def sales():

return ("...")

This example has been taken from this page: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-python-ref.html

By the way, be careful about how it is generated regarding you are dealing with a streaming table or not: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-faqs-issues.html#how-do-i-...

HTH

Laurent

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.