cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

DLT pipeline

Rishitha
New Contributor III

Hi all!

I have a question about setting a target schema. How to set different targets for 2 different tables in the same delta live table pipeline. 
We have 2 target schemas in a database Bronze_chema and silver_schema. 
 
The pipeline has a streaming raw table (Bronze table) Table A and a processed table (Silver Table) derived from the bronze table,  Table B. When I set the target schema in the Workflow settings it defaults both the tables to that target schema. 
Table A -> Bronze_schema
TableB -> Bronze_schema
 
But I want  
Table A -> Bronze_schema and 
Table B -> Silver_schema, as part of the same delta live table pipeline.  please suggest how to set the targets as part of pipeline code
1 ACCEPTED SOLUTION

Accepted Solutions

btafur
Contributor III
Contributor III

At the moment, with the current features available, it is not possible to have multiple schemas in one DLT pipeline. In some cases, it is possible to achieve something similar by creating multiple DLT pipelines for the different schemas.

View solution in original post

3 REPLIES 3

Rishitha
New Contributor III

I have also asked the question in the office hours, they said it will be a new feature, announced in the summit too, but there is a workaround to do it now. Anyone implemented the workaround? '

 

btafur
Contributor III
Contributor III

At the moment, with the current features available, it is not possible to have multiple schemas in one DLT pipeline. In some cases, it is possible to achieve something similar by creating multiple DLT pipelines for the different schemas.

Rishitha
New Contributor III

Thanks again @btafur 

Hoping for this feature to release soon!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.