- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-07-2025 08:39 PM - edited 03-07-2025 08:41 PM
Hi @BobCat62 ,
So the thing is Now dlt has different modes dlt direct publishing mode , classic mode(legacy). Look here for mode details : https://docs.databricks.com/aws/en/release-notes/product/2025/january#dlt-now-supports-publishing-to...
1. if you are using legacy mode in dlt configuration setting { target variable will be defined(basically the default schema of the pipeline)}, so if using this method dlt expects you to use live.pumpdata_silver on your table where you want it to be dependent on the first pumpdata_bronze table. It makes sure that refreshing of the dependent table starts only when the bronze refreshing is done hence, the latest records.
Though above method is legacy now. Its a best practice to follow latest advancements.
2. dlt direct publishing mode, in your dlt pipeline configuration (if you use schema var instead of target var (both have same use but are mutually exclusive only one can be used) , then it automatically means your pipeline is in latest mode hence live is not required and dlt will automatically handle all the dependencies itself.
I haven't used sequentialityin direct publishing moe but the above link would have some guidelines on it.