Implementing Databricks Persona in

d_kailthya — Sat, 21 Sep 2024 12:19:39 GMT

Hi all,

I am looking to implement the "persona" based access control across multiple workspaces for multiple user groups in Azure Databricks workspaces. Specifically,

- I have a "DEV" workspace where the developer groups (Data Engineers and ML Engineers) should be able to start up compute clusters and execute Databricks workflows and jobs at will

- I have a "PROD" workspace where the developer groups should only be able to view the workflow or job details and logs but should not be able to execute these.

To me it seems like the "persona" based solution described in this article fits the bill nicely. Does anyone have a suggestion on how to implement this in terraform, e.g. using the Databricks provider ?

Thanks in advance!

Dipanjan

Re: Implementing Databricks Persona in

mark_ott — Mon, 03 Nov 2025 21:26:26 GMT

You can implement persona-based access control for Azure Databricks workspaces using Terraform and the Databricks provider, aligning with the setup you described for DEV and PROD environments. Terraform allows you to codify workspace configuration, user/group assignments, and access controls so that developer groups have appropriate permissions in each workspace.

DEV Workspace: Granting Execution Privileges

For DEV, you want Data Engineers and ML Engineers to manage clusters and run jobs freely. You should:

Assign these groups Databricks roles such as "Can Manage" at both cluster and job/workflow levels.
Use Terraform databricks_group resource to manage user groups.
Assign permissions using databricks_permissions for clusters and jobs.

Sample Terraform configuration:

text

resource "databricks_group" "data_engineers" {
  display_name = "data-engineers"
}

resource "databricks_group" "ml_engineers" {
  display_name = "ml-engineers"
}

# Assign cluster management permissions
resource "databricks_permissions" "dev_cluster_permissions" {
  cluster_id = databricks_cluster.dev.id

  access_control {
    group_name = databricks_group.data_engineers.display_name
    permission_level = "CAN_MANAGE"
  }

  access_control {
    group_name = databricks_group.ml_engineers.display_name
    permission_level = "CAN_MANAGE"
  }
}

# Assign job execution permissions
resource "databricks_permissions" "dev_job_permissions" {
  job_id = databricks_job.dev.id

  access_control {
    group_name = databricks_group.data_engineers.display_name
    permission_level = "CAN_MANAGE"
  }

  access_control {
    group_name = databricks_group.ml_engineers.display_name
    permission_level = "CAN_MANAGE"
  }
}

This configuration ensures both groups can start clusters and execute jobs in DEV.

PROD Workspace: Granting Read-Only Access

For PROD, restrict these groups so they can only view job/workflow details and logs, not execute them.

Assign them "Can View" permissions only—using the same resources, but with "CAN_VIEW" instead of "CAN_MANAGE".

Sample Terraform configuration:

text

resource "databricks_permissions" "prod_cluster_permissions" {
  cluster_id = databricks_cluster.prod.id

  access_control {
    group_name = databricks_group.data_engineers.display_name
    permission_level = "CAN_VIEW"
  }

  access_control {
    group_name = databricks_group.ml_engineers.display_name
    permission_level = "CAN_VIEW"
  }
}

resource "databricks_permissions" "prod_job_permissions" {
  job_id = databricks_job.prod.id

  access_control {
    group_name = databricks_group.data_engineers.display_name
    permission_level = "CAN_VIEW"
  }

  access_control {
    group_name = databricks_group.ml_engineers.display_name
    permission_level = "CAN_VIEW"
  }
}

This aligns with a persona-based approach, where personas (developer, analyst, etc.) map to groups that receive distinct role assignments per workspace.

Additional Guidance

Use Terraform workspaces or separate configurations for DEV and PROD environments to keep isolation.
Consult the Databricks provider documentation for the latest permission levels and supported resources.
Consider using Databricks access control lists (ACLs), cluster policies, and workspace-level role assignments for more granular control.

References

Persona-based access is described in detail in Databricks documentation.
The Terraform Databricks provider gives exact implementation examples for roles, permissions, and resource management.