cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Adding new columns to a Delta Live table in a CDC process

ameya
New Contributor

Hi 

I am new to databricks and still learning.

I am trying to do a CDC on a table.

 

 

APPLY CHANGES INTO LIVE.table1
FROM schema2.table2
KEYS (Id)
SEQUENCE BY orderByColumn
COLUMNS * EXCEPT (col1, col2)
  STORED AS SCD TYPE 1 ;

 

 

table1 is in schema1 and is silver layer. table2 is in schema2 and is bronze layer.

  1. table2 may have different, new columns than table1. how do I insert or update these new columns into table1. I am assuming that would give us schema mismatch errors.
  2. I want to insert some new columns based on the whether the operation is an insert operation or an update operation. If it is an update operation, I want to insert a column called 'LastModifiedDate' with the current timestamp and if it is an insert operation I want to insert a column called 'CreatedDate' as current timestamp.

How can I do these 2 things?

 

Thanks in advance.

0 REPLIES 0
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!