cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Dev/Prod Environments in AWS: Separate Accounts vs. Separate Workspaces?

tana_sakakimiya
New Contributor III

Hello everyone,

I'm looking for some advice on best practices. When setting up development and production environments for Databricks on AWS, is it better to use completely separate AWS accounts, or is it sufficient to use separate workspaces within a single AWS account?

Intuitively, seperating AWS account is more secured but does it overkill and generate overcomplexity when i implement DABs?

What is the most common approach?

Thanks!

1 REPLY 1

szymon_dybczak
Esteemed Contributor III

Hi @tana_sakakimiya ,

 

I allow myself copy and paste brilliant answer on similar question provided by user Isi:

"Option A: Multiple Databricks accounts and multiple AWS accounts

This model offers the highest level of isolation. Each environment lives in its own Databricks and AWS account, allowing for complete separation of resources, users, and billing. Itโ€™s good if you are a large organization. But itโ€™s also the most expensive and complex to maintain, since it involves duplicating configurations, user management, pipelines, and access control. I wouldnโ€™t recommend this option unless you have very strong isolation or compliance requirements.

 

Option B: A single Databricks account with multiple workspaces and multiple AWS accounts

It enables environment separation at the infrastructure level using different AWS accounts, while still maintaining centralized management of workspaces under a single Databricks account. It strikes a balance between isolation and operational efficiency. I would recommend this setup because itโ€™s an easy way to work: you just need to manage cross-account access in AWS, and everything stays centralized on the Databricks side.

 

Option C: A single Databricks account with multiple workspaces and a single AWS account

This setup is common in small to medium companies. Isolation is achieved through separate networks (e.g., different subnets or VPCs), IAM policies, and logical separation of data using S3 buckets or Unity Catalog. Itโ€™s a simple, effective, and easy-to-manage approach. While it doesnโ€™t offer the same level of isolation as the previous options, itโ€™s often more than enough for most use cases.

If youโ€™re a small company and donโ€™t have very strict isolation needs, Iโ€™d go with this option. Just keep in mind that, since everything lives in a single AWS account, youโ€™ll need to clearly separate whatโ€™s dev/pre/pro and put extra care into access policies to ensure everything is properly segmented."

Source: Solved: Best Practices for Mapping Between Databricks and ... - Databricks Community - 120788