Audit Access Rights

[Databricks1126]

We have a large Databricks instance, and we are performing a technical audit of Databricks to identify (1) the full list of users, service principals, and groups; (2) the full list of objects (e.g. catalogs, schemas, jobs, notebooks, etc.); and (3) the access levels of the users, service principals, and groups to those objects.

Here are the specific asks:

• What are all the universe of ‘objects’ on Databricks that users can create and use to transform data? (e.g. catalogs, schemas, jobs, notebooks, etc.)
• Are there hierarchal access relationships between these objects? For example, access to Object A gives you access to Object B and Object C.
• How can we pull this information programmatically from Databricks?

[WiliamRosa]

Hi @Databricks1126,

I understand that you’re looking to capture permissions across a wide variety of Databricks objects. These can generally be grouped into three main categories:

- Data objects (Unity Catalog–governed) – catalogs, schemas, tables, views, volumes, functions, models.
- Workspace objects (compute / code / workflow) – jobs, notebooks, repos, pipelines, SQL warehouses, dashboards.
- Identity / configuration objects – users, service principals, groups, secrets, clusters, instance pools.

Because this is quite a broad universe, a good first step for such an audit is to use the Databricks REST API. The official reference is here:
https://docs.databricks.com/api/workspace/introduction

For example, you can start by retrieving the full list of workspace users via the SCIM API, and then for each user (by ID or email) check their associated permissions:

import requests, json

host = spark.conf.get("spark.databricks.workspaceUrl")
token = dbutils.secrets.get("my-scope", "DATABRICKS_TOKEN")

# List all users
url = f"https://{host}/api/2.0/preview/scim/v2/Users"
resp = requests.get(url, headers={"Authorization": f"Bearer {token}"})
resp.raise_for_status()
data = resp.json()

for user in data.get("Resources", []):
    print(user["id"], user["userName"], user.get("displayName"))

# Lookup a specific user by email
user_email = "user@test.com"
url = f"https://{host}/api/2.0/preview/scim/v2/Users?filter=userName eq \"{user_email}\""
resp = requests.get(url, headers={"Authorization": f"Bearer {token}"})
print(json.dumps(resp.json(), indent=2))

The response is a SCIM User document (e.g., id, userName, displayName, groups, entitlements, …).
Hope that helps!