Dev/Prod Environments in AWS: Separate Accounts vs. Separate Workspaces?

[tana_sakakimiya]

Hello everyone,

I'm looking for some advice on best practices. When setting up development and production environments for Databricks on AWS, is it better to use completely separate AWS accounts, or is it sufficient to use separate workspaces within a single AWS account?

Intuitively, seperating AWS account is more secured but does it overkill and generate overcomplexity when i implement DABs?

What is the most common approach?

Thanks!

[szymon_dybczak · Accepted solution]

Hi @tana_sakakimiya ,

I allow myself copy and paste brilliant answer on similar question provided by user Isi:

"Option A: Multiple Databricks accounts and multiple AWS accounts

This model offers the highest level of isolation. Each environment lives in its own Databricks and AWS account, allowing for complete separation of resources, users, and billing. It’s good if you are a large organization. But it’s also the most expensive and complex to maintain, since it involves duplicating configurations, user management, pipelines, and access control. I wouldn’t recommend this option unless you have very strong isolation or compliance requirements.

Option B: A single Databricks account with multiple workspaces and multiple AWS accounts

It enables environment separation at the infrastructure level using different AWS accounts, while still maintaining centralized management of workspaces under a single Databricks account. It strikes a balance between isolation and operational efficiency. I would recommend this setup because it’s an easy way to work: you just need to manage cross-account access in AWS, and everything stays centralized on the Databricks side.

Option C: A single Databricks account with multiple workspaces and a single AWS account

This setup is common in small to medium companies. Isolation is achieved through separate networks (e.g., different subnets or VPCs), IAM policies, and logical separation of data using S3 buckets or Unity Catalog. It’s a simple, effective, and easy-to-manage approach. While it doesn’t offer the same level of isolation as the previous options, it’s often more than enough for most use cases.

If you’re a small company and don’t have very strict isolation needs, I’d go with this option. Just keep in mind that, since everything lives in a single AWS account, you’ll need to clearly separate what’s dev/pre/pro and put extra care into access policies to ensure everything is properly segmented."

Source: Solved: Best Practices for Mapping Between Databricks and ... - Databricks Community - 120788