topic Re: Clone the data from Prod Workspace to Test workspace in Data Engineering

Clone the data from Prod Workspace to Test workspace

Databricks9876 — Tue, 22 Feb 2022 15:09:54 GMT

Hi there,

I have requirement to deep clone the data from Prod workspace to Test Workspace.

I know we can do in the same workspace, but I am not aware of between workspaces.

I heard some key is needed and have some firewall issues when linking between two work spaces.

Really appreciate if someone could shed some light .

Thank you,

Rup

Re: Clone the data from Prod Workspace to Test workspace

Anonymous — Tue, 22 Feb 2022 18:17:22 GMT

Hello, Rup aka @swarup muthumula - It's nice to meet you. Welcome to the community and thank you for bringing us this question. Let's give it a while for the members to respond. We'll come back if we need to.

Re: Clone the data from Prod Workspace to Test workspace

-werners- — Wed, 23 Feb 2022 08:41:32 GMT

Can I ask what you mean by data? Is it your physical storage (data lake) or the notebooks etc?

Re: Clone the data from Prod Workspace to Test workspace

Atanu — Tue, 08 Mar 2022 13:51:23 GMT

@swarup muthumula are you looking for entire workspace migration ? which cloud you are in?

Re: Clone the data from Prod Workspace to Test workspace

Databricks9876 — Wed, 09 Mar 2022 01:08:58 GMT

Hi Werners,

Thanks for your reply and Apologies for my late reply.

Yes, I am looking at physical storage. We have created tables and the data is loading through Notebooks. I don't want to run the Notebooks again in UAT, instead I want to clone the PROD data into UAT. The PROD is updated daily so whenever I need to I wan to clone those tables. We already have objects created in UAT

Re: Clone the data from Prod Workspace to Test workspace

Databricks9876 — Wed, 09 Mar 2022 01:10:25 GMT

Hi Atanu,

Thanks for your reply.

I am in Azure and want to just clone only data not even workspace and Notebooks

Re: Clone the data from Prod Workspace to Test workspace

jose_gonzalez — Tue, 05 Apr 2022 23:35:55 GMT

Hi @swarup muthumula ,

Are you trying to clone Delta tables? if you do, you can use delta clone. Docs here https://docs.microsoft.com/en-us/azure/databricks/delta/delta-utility#clone-delta-table

Re: Clone the data from Prod Workspace to Test workspace

Databricks9876 — Wed, 06 Apr 2022 00:58:41 GMT

Thank you , Jose. Appreciate your reply.

I think the cloning is done at the same workspace level. Can we also do across workspaces? Can i Give the same clone command in another workspace by giving the location of the target table directing to Other workspace?

Re: Clone the data from Prod Workspace to Test workspace

Atanu — Sun, 17 Apr 2022 18:00:26 GMT

@swarup muthumula Azure workspace has self service migration