cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

CDC DLT

Stellar
New Contributor II

Hi all,

I would appreciate some clarity regarding the DLT and CDC. So my first question would be, when it comes to the "source" table in the synta, is that CDC table or? Further, if we want to use only databricks, would mounting foreign catalog be a good thing? Also, what would be some of the solutions you guys implemented to initially pull in the data? I need to pull in certain portion of the data not the whole table and then continue with DLT CDC.

Also, if my CDC table is in database(SQL Server), what would be the preferred method to connect to that database and read cdc table from Databricks?

Thanks!

#CDC #Databricks

1 REPLY 1

Stellar
New Contributor II

@Retired_mod Thanks for the explanations! What I am not sure about is the following. I create streaming table within the apply changes syntax. However I need to have a table already with the data. Does this mean that in the apply changes, create streaming table syntax is just refrencing existing table as delta live table or? Further, do we always load the whole cdc table? Sorry if my questions are not good but I am trying to figure this out. Also just to confirm my understanding these keys are not from the cdc table but rather from the table we wish to apply changes to because we need to connect two records together. 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group