cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Identity column in DLT using Python

kskistad
New Contributor III

How would I implement the Identity column in Delta Live Tables using Python syntax?

GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( [ START WITH start ] [ INCREMENT BY step ] ) ]

 

1 ACCEPTED SOLUTION

Accepted Solutions

LaurentLeturgez
Contributor

Hi @Kory Skistadโ€‹ 

Please find below the table schema definition to use in a python dlt pipeline. You can see it mentions the identity column definition.

@dlt.table(

comment="Raw data on sales",

schema="""

customer_id STRING,

customer_name STRING,

number_of_line_items STRING,

order_datetime STRING,

order_number LONG,

order_day_of_week STRING GENERATED ALWAYS AS (dayofweek(order_datetime))

""",

partition_cols = ["order_day_of_week"])

def sales():

return ("...")

This example has been taken from this page: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-python-ref.html

By the way, be careful about how it is generated regarding you are dealing with a streaming table or not: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-faqs-issues.html#how-do-i-...

HTH

Laurent

View solution in original post

1 REPLY 1

LaurentLeturgez
Contributor

Hi @Kory Skistadโ€‹ 

Please find below the table schema definition to use in a python dlt pipeline. You can see it mentions the identity column definition.

@dlt.table(

comment="Raw data on sales",

schema="""

customer_id STRING,

customer_name STRING,

number_of_line_items STRING,

order_datetime STRING,

order_number LONG,

order_day_of_week STRING GENERATED ALWAYS AS (dayofweek(order_datetime))

""",

partition_cols = ["order_day_of_week"])

def sales():

return ("...")

This example has been taken from this page: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-python-ref.html

By the way, be careful about how it is generated regarding you are dealing with a streaming table or not: https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-faqs-issues.html#how-do-i-...

HTH

Laurent

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group