You can implement persona-based access control for Azure Databricks workspaces using Terraform and the Databricks provider, aligning with the setup you described for DEV and PROD environments. Terraform allows you to codify workspace configuration, user/group assignments, and access controls so that developer groups have appropriate permissions in each workspace.
DEV Workspace: Granting Execution Privileges
For DEV, you want Data Engineers and ML Engineers to manage clusters and run jobs freely. You should:
-
Assign these groups Databricks roles such as "Can Manage" at both cluster and job/workflow levels.
-
Use Terraform databricks_group resource to manage user groups.
-
Assign permissions using databricks_permissions for clusters and jobs.
Sample Terraform configuration:
resource "databricks_group" "data_engineers" {
display_name = "data-engineers"
}
resource "databricks_group" "ml_engineers" {
display_name = "ml-engineers"
}
# Assign cluster management permissions
resource "databricks_permissions" "dev_cluster_permissions" {
cluster_id = databricks_cluster.dev.id
access_control {
group_name = databricks_group.data_engineers.display_name
permission_level = "CAN_MANAGE"
}
access_control {
group_name = databricks_group.ml_engineers.display_name
permission_level = "CAN_MANAGE"
}
}
# Assign job execution permissions
resource "databricks_permissions" "dev_job_permissions" {
job_id = databricks_job.dev.id
access_control {
group_name = databricks_group.data_engineers.display_name
permission_level = "CAN_MANAGE"
}
access_control {
group_name = databricks_group.ml_engineers.display_name
permission_level = "CAN_MANAGE"
}
}
This configuration ensures both groups can start clusters and execute jobs in DEV.
PROD Workspace: Granting Read-Only Access
For PROD, restrict these groups so they can only view job/workflow details and logs, not execute them.
-
Assign them "Can View" permissions only—using the same resources, but with "CAN_VIEW" instead of "CAN_MANAGE".
Sample Terraform configuration:
resource "databricks_permissions" "prod_cluster_permissions" {
cluster_id = databricks_cluster.prod.id
access_control {
group_name = databricks_group.data_engineers.display_name
permission_level = "CAN_VIEW"
}
access_control {
group_name = databricks_group.ml_engineers.display_name
permission_level = "CAN_VIEW"
}
}
resource "databricks_permissions" "prod_job_permissions" {
job_id = databricks_job.prod.id
access_control {
group_name = databricks_group.data_engineers.display_name
permission_level = "CAN_VIEW"
}
access_control {
group_name = databricks_group.ml_engineers.display_name
permission_level = "CAN_VIEW"
}
}
This aligns with a persona-based approach, where personas (developer, analyst, etc.) map to groups that receive distinct role assignments per workspace.
Additional Guidance
-
Use Terraform workspaces or separate configurations for DEV and PROD environments to keep isolation.
-
Consult the Databricks provider documentation for the latest permission levels and supported resources.
-
Consider using Databricks access control lists (ACLs), cluster policies, and workspace-level role assignments for more granular control.
References
-
Persona-based access is described in detail in Databricks documentation.
-
The Terraform Databricks provider gives exact implementation examples for roles, permissions, and resource management.
By following these steps, you'll maintain strict boundaries between DEV and PROD, in line with persona-based governance principles.
