cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How to define write Option in a DLT using Python?

pablociu
New Contributor

In a normal notebook I would save metadata to my Delta table using the following code:

(
  df.write
    .format("delta")
    .mode("overwrite")
    .option("userMetadata", user_meta_data) 
    .saveAsTable("my_table")
)

But I couldn't find online how can I set up the '.option("userMetadata...' in a DLT. Is there a document I couldn't find that would explain how to set up this?

2 REPLIES 2

jose_gonzalez
Moderator
Moderator

I would like to share the following docs page https://docs.databricks.com/delta-live-tables/python-ref.html

United_Communit
New Contributor II

In Delta lab you can set up User MetaData so i will give you some tips

from delta import DeltaTable

# Create or load your Delta table

delta_table = DeltaTable.forPath(spark, "path_to_delta_table")

# Define your user metadata myccpay

user_meta_data = {"key1": "value1", "key2": "value2"}

# Set the userMetadata option in your Delta table write operation

delta_table.alias("df").merge(

  df.alias("new_data"),

  "df.id = new_data.id"

 ).whenMatchedUpdateAll()

 .whenNotMatchedInsertAll()

 .option("userMetadata", user_meta_data)

 .execute()

you should try this code.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.