cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Blue/Green Deployment, Table Cloning, and Delta live table pipelines

6502
New Contributor III

this a rather complex question that addresses Databricks users only. Let me recap a bit of the context that produced it. In the attempt to adopt the Blue/Green deployment protocol, we found good applications of the table cloning capabilities offered by Databricks. At every new deployment interaction, we clone the existing table into the new system while keeping the old one running. After the new system has been validated, we update the views so that anything works with no interruption. Well, but how do we "clone" the tables used in DLT pipelines? Any help or suggestion is appreciated!

1 REPLY 1

Kaniz
Community Manager
Community Manager

Hi @6502To clone tables used in Databricks Delta Lake (DLT) pipelines, you can utilize Delta Lake's versioning and time travel features.

Here are the steps you can follow:

1. First, identify the version of the table you want to clone. This could be the latest version or a specific version your pipeline uses.

2. Use the Delta Lake's CLONE command to clone the table. The syntax is as follows:

python
CLONE source_table [VERSION AS OF version | TIMESTAMP AS OF timestamp] AS destination_table

Where:
source_table Is the name of the table you want to clone?
version Is the version number of the table you want to clone?
timestamp It is the timestamp of the table version you want to clone.
destination_table This is the name of the new table.

Please note that the CLONE the command is not directly supported in Databricks. However, you can achieve similar functionality by using Delta Lake's time travel feature to read a specific table version and then write it to a new table.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.