Blue/Green Deployment, Table Cloning, and Delta live table pipelines

[6502]

this a rather complex question that addresses Databricks users only. Let me recap a bit of the context that produced it. In the attempt to adopt the Blue/Green deployment protocol, we found good applications of the table cloning capabilities offered by Databricks. At every new deployment interaction, we clone the existing table into the new system while keeping the old one running. After the new system has been validated, we update the views so that anything works with no interruption. Well, but how do we "clone" the tables used in DLT pipelines? Any help or suggestion is appreciated!

[Kaniz_Fatma]

Hi @6502, To clone tables used in Databricks Delta Lake (DLT) pipelines, you can utilize Delta Lake's versioning and time travel features.

Here are the steps you can follow:

1. First, identify the version of the table you want to clone. This could be the latest version or a specific version your pipeline uses.

2. Use the Delta Lake's CLONE command to clone the table. The syntax is as follows:

python
CLONE source_table [VERSION AS OF version | TIMESTAMP AS OF timestamp] AS destination_table

Where:
- source_table Is the name of the table you want to clone?
- version Is the version number of the table you want to clone?
- timestamp It is the timestamp of the table version you want to clone.
- destination_table This is the name of the new table.

Please note that the CLONE the command is not directly supported in Databricks. However, you can achieve similar functionality by using Delta Lake's time travel feature to read a specific table version and then write it to a new table.