What is the best practice for managing different environments (Staging vs Production) on Databricks?

[Anonymous]

Should we create separate workspaces for Dev/Test/Prod ? Or should we have 1 workspace and create separate folders for Dev/Test/Prod?

[Srikanth_Gupta_ · Accepted solution]

as per my previous experience, its always good to have different workspaces for different envs, its easy to maintain and helps better with CICD pipeline as well, because lot of organizations provide deployment access to Developers in Dev env but not in UAT and Prod environments.

[Oliver_Angelil]

@Srikanth Anumula​ we currently have 2 databricks workspaces: development and production. If the data on both are the same and not sensitive, is there any reason to have two workspaces? One could have one and simply have different git branches: feature-branches, devevelopment-branch, production-branch. I believe that might simplify things... .WDYT?

[harraz]

Hi Srikanth,

Can you elaborate on your answer a bit more. We are new to databricks and we are not sure how can a developer work on a notebook that changes a table and at the same time make sure they don't make changes to production tables unless the code is tested and approved. Is there a best practice page that shows how to setup workspaces for this use case? And when I say workspace I mean the the personal workspace where developers have their own notebooks and hive metastore

thanks