cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How to compare notebooks for deployment

BeginnerBob
New Contributor III

I have 2 Azure Databricks workspaces (test and prod)

I want to compare the notebooks to see the changes.

Are there any tools which can help do this?

Thanks,

3 REPLIES 3

Hubert-Dudek
Esteemed Contributor III

Yes please use Repos and set git repositories so that you can use the standard git process during merges.

BeginnerBob
New Contributor III

Thanks Hubert.

What's the best approach here? Do I have 2 trunks in my repos (1 for dev and the other for production)?

Can I do a compare between the 2 trunks to generate all the notebooks which need to be deployed. How do I do this compare (in DevOps, vsCode etc.)

What is the best way to do this deployment i.e. running Powershell scripts, manual overwrite etc.

Thanks,

Hubert-Dudek
Esteemed Contributor III

I would set up two separate branches (one on each of your workspace), push from them, and then merge those branches to the "dev" or "prod" branch in the external tool (by creating a pull request). My favorite is Azure DevOps, but the most popular are GitHub and Gitlab.

Of course, you can have as many branches as you want in a separate folder in each workspace. So you can have on your workspace branch dedicated to that workspace as well "dev" and "prod", which are created after merging pull requests and are the same in both workspaces (so you don't develop on "dev" and "prod" branches, you make only pull to get them).

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.