cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

DLT pipeline

Rishitha
New Contributor III

Hi all!

I have a question about setting a target schema. How to set different targets for 2 different tables in the same delta live table pipeline. 
We have 2 target schemas in a database Bronze_chema and silver_schema. 
 
The pipeline has a streaming raw table (Bronze table) Table A and a processed table (Silver Table) derived from the bronze table,  Table B. When I set the target schema in the Workflow settings it defaults both the tables to that target schema. 
Table A -> Bronze_schema
TableB -> Bronze_schema
 
But I want  
Table A -> Bronze_schema and 
Table B -> Silver_schema, as part of the same delta live table pipeline.  please suggest how to set the targets as part of pipeline code
1 ACCEPTED SOLUTION

Accepted Solutions

btafur
Databricks Employee
Databricks Employee

At the moment, with the current features available, it is not possible to have multiple schemas in one DLT pipeline. In some cases, it is possible to achieve something similar by creating multiple DLT pipelines for the different schemas.

View solution in original post

3 REPLIES 3

Rishitha
New Contributor III

I have also asked the question in the office hours, they said it will be a new feature, announced in the summit too, but there is a workaround to do it now. Anyone implemented the workaround? '

 

btafur
Databricks Employee
Databricks Employee

At the moment, with the current features available, it is not possible to have multiple schemas in one DLT pipeline. In some cases, it is possible to achieve something similar by creating multiple DLT pipelines for the different schemas.

Rishitha
New Contributor III

Thanks again @btafur 

Hoping for this feature to release soon!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group