Best Practices for Managing Permissions in Databricks Workspaces

[oliviaanne]

Hi everyone,

I’m interested in learning how other teams manage user permissions and access control in Databricks workspaces, especially when multiple users and groups need different levels of access.

For example, when working with SQL Warehouses, Unity Catalog, and hared data resources, what approach do you find most effective for keeping permissions organized and secure?

Do you prefer managing permissions directly through the Databricks UI, using REST APIs, or automating everything with Terraform?

I’d love to hear about your experiences, any challenges you’ve faced, and the best practices you would recommend for making permission management easier as a team grows.

Thanks in advance for sharing your insights!

[Aravind_Reddy]

Hello @oliviaanne 

In my experience, a scalable approach to managing permissions in Databricks is to combine group-based access control, centralized permission management, and regular access reviews rather than managing permissions individually through the UI.

A few best practices I would recommend:

Use groups instead of individual users: Manage access through identity-provider groups synchronized with Databricks. This simplifies onboarding, offboarding, and periodic permission reviews.

Separate workspace and data permissions: Use workspace ACLs to control access to notebooks, jobs, and SQL Warehouses, while using Unity Catalog grants to govern access to catalogs, schemas, tables, and other data assets. Both layers are important.

Use consistent permission management practices: Establish standard access roles and permission templates for different teams and environments. This helps maintain consistency across Dev, Test, and Prod and reduces the risk of configuration differences.

Use the UI and REST APIs appropriately: The UI is useful for initial configuration and troubleshooting, while Databricks REST APIs can support custom automation and integrations when managing permissions at scale.

Apply the principle of least privilege: Grant users only the permissions they need, reserve administrative privileges for designated groups, and use service principals for automated production workloads where appropriate.

Audit permissions regularly: Review direct and inherited permissions, remove stale access, and verify that production resources and sensitive data are not exposed more broadly than intended.

One additional recommendation is to establish a clear ownership and approval process for permission changes, especially in production environments.

For growing teams, I would focus on group-based access, standardized permission practices, API-driven automation where needed, and regular access reviews. The key is to keep permissions consistent, auditable, and easy to maintain as the organization scales.

[SumeshKashyap]

Adding the "how" to the points above. What has scaled well for us is to grant only to account-level groups synced from the IdP, never to individual users. We manage both layers in Terraform: Unity Catalog grants for data, and workspace ACLs for warehouses, jobs and clusters. Then we use system tables to catch anything granted outside the code.

1. Groups from the IdP

Sync groups with SCIM, or with automatic identity management on Azure, at the account level and assign them to workspaces. Onboarding and offboarding then happen in Entra or Okta, and nothing changes in Databricks. Use service principals for jobs and CI.

2. Grants as code

resource "databricks_grants" "sales" {

  schema = "main.sales"

  grant {

    principal  = "data-analysts"

    privileges = ["USE_SCHEMA", "SELECT"]

  }

  grant {

    principal  = "data-engineers"

    privileges = ["USE_SCHEMA", "SELECT", "MODIFY", "CREATE_TABLE"]

  }

}

resource "databricks_permissions" "bi_warehouse" {

  sql_endpoint_id = databricks_sql_endpoint.bi.id

  access_control {

    group_name       = "data-analysts"

    permission_level = "CAN_USE"

  }

}

databricks_grants is authoritative for that securable: on the next apply it removes any grant not in the code, which is what keeps UI changes from drifting. Use the singular databricks_grant if other teams also grant on the same object. Remember the group also needs USE_CATALOG on the parent catalog.

3. Audit for drift

-- direct grants to individual users (should return nothing)

SELECT grantee, table_catalog, table_schema, table_name, privilege_type

FROM system.information_schema.table_privileges

WHERE grantee LIKE '%@%';

-- who changed UC permissions recently

SELECT event_time, user_identity.email,

       request_params.securable_full_name, request_params.changes

FROM system.access.audit

WHERE service_name = 'unityCatalog' AND action_name = 'updatePermissions'

  AND event_date >= current_date() - 30;

Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/ and https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grants 

[balajij8]

You can follow the standard principle of managing access at scale by assigning permissions exclusively to groups rather than individuals. By leveraging identity federation, you can sync these groups directly from the upstream identity provider like Microsoft Entra. It makes onboarding and offboarding completely seamless - when someone's group membership changes in the IdP, their workspace access automatically updates based on it. You should ensure that powerful roles like account, workspace or metastore admins, as well as core privileges like MANAGE or ALL PRIVILEGES, are granted only to groups. For production, it is critical to reassign object ownership from individual creators to a designated service group.

For the data resources, Unity Catalog needs to be your single governance backbone. Because it uses a hierarchical securable model - from metastore to catalog to schema down to tables, views or volumes - privileges like SELECT, MODIFY and USE SCHEMA generally inherit downward. It lets you grant broad access at the catalog or schema level instead of managing per table ACLs. Beyond standard privilege grants, you can lean on Unity Catalog for attribute-based access control (ABAC) using governed tags, row and column level masking for sensitive data and workspace level restrictions to strictly isolate where data is accessible. When using SQL Warehouses, the valid approach is to have workspace admins create and configure the compute centrally, forcing all data access to flow through Unity Catalog rather than relying on legacy workspace-level table ACLs.

As your team grows, the mechanics of how you apply these permissions will need to evolve. The Databricks UI via Catalog Explorer and Admin Settings is generally fine for small teams or ad hoc exploration. Once you start scaling across multiple workspaces or environments, you must transition to infrastructure as code. Using the Databricks Terraform Provider or DAB is the standard here - it supports nearly all Databricks APIs, including Unity Catalog grants, workspace object ACLs and identity configuration. It allows you to manage permissions as version controlled reviewable code. When setting up this automation, CI/CD pipelines or Terraform authentication, ensure you use service principals rather than using a user's personal access token.