How do you create a Sandbox in your data environment ?

[William_Scardua]

Hi guys,

How do you create a Sandbox in your data environment ? have any idea ?

Azzure/AWS + Data Lake + Databricks

[Hubert-Dudek · Accepted solution]

maybe use Azure DataFactory to periodically copy your data to sandbox storage.

If you need to configure mounts and sql databases/tables the best is to have notebook to do that which is run in both environments with widget which specify deployment (so it will replace mount and table location accordingly).

1) copy in dataFactory

2) when completed run deployment notebook (also can be with dataFactory and param available though widget can be set there)

[William_Scardua]

thank you @Hubert Dudek​ 

[-werners-]

Depends on how much you want the sandbox to be disconnected from the rest.

The ideal scenario is a complete separate setup, as in DEV-QA-PRD-SANDBOX.

But to be honest, I think that is overkill.

If you have a separate storage account or you have a separate BLOB or even a subdirectory which is protected with permissions, you have already quite a lot.

Because the data is the most important part.

Then the notebooks: you can opt for a separate databricks account but again, you can do without.

f.e. use Repos for your 'official' notebooks, and the workspace/user folder for playing around.

You only have to make sure that you use the correct mount and that can be set with a widget as Hubert mentioned.

It also depends on the amount of people working on databricks. If you are only a small team you do not have to be too strict. But with lots of people and frequent personnel changes (consultants f.e.) it is a good idea to have strict permissions/procedures etc.

[William_Scardua]

I agree @Werner Stinckens​ , thank you

[missyT]

In a sandbox environment, you will find the Designer enabled. You can activate Designer by selecting the design icon Designer. on a page, or by choosing the Design menu item in the Settings Settings menu.

[William_Scardua]

Thank you @Missy Trussell​