What Is ABAC and Why Does It Matter?
Attribute-Based Access Control (ABAC) is a data governance model now available in Databricks, designed to offer fine-grained, dynamic, and scalable access control for data, AI assets, and files managed through Databricksโ Unity Catalog. While currently in Beta for both AWS and Azure Databricks, ABAC builds onโand coexists withโthe existing Unity Catalog (UC) security model, adding powerful new capabilities for secure and compliant data management.
With ABAC, access is governed not just by static roles, but by the broader attributes of users, data, and the environment. These attributes can include user role, department, region, time, or custom tags attached to data assets. The system allows administrators to define policies that adapt automatically as attributes or business contexts change.
Key ABAC Components
- Attributes: Descriptive metadata about users, data, or requests. Examples: region, sensitivity level, department, time, or custom-defined tags.
- Policies: Conditional rules that determine access; can use any combination of attributes to allow or restrict access to specific data assets.
- Inheritance: Policies and tags propagate through the object hierarchy (catalog โ schema โ table), simplifying and streamlining management.
- RLS/CLF Functions: SQL user-defined functions (UDFs) that enforce row-level security (RLS) and column-level filtering/masking (CLF) as per assigned tags.
- Rules: ABAC applies rules to specific users or groups, dynamically evaluating the defined attributes and policies.
How ABAC Works: Benefits Over Traditional Models
Benefit | Traditional Limitations | How ABAC Helps Now |
User Attribute Access | Hard to restrict access by region/dept | ABAC uses attributes (e.g., region = "EU") for contextual access |
Row-Level Security | No row-level filteringโsame view for all | Users see only allowed rows (e.g., India team sees only India data) |
Column-Level Security | Sensitive columns always visible by role | Hides sensitive columns based on user/data classification |
Dynamic Policies | Static rules, manual updates needed | Policies adjust with user/data changes automatically |
Fewer Roles Needed | Role explosion for teams, regions | One role serves many users via attribute conditions |
Compliance & Privacy | Custom logic needed; limited to UC | ABAC enforces rules directly within Unity Catalog |
Scalable Security | Hard to manage at enterprise scale | Scales via metadata/tags, not via multiplying roles |
Tags: The Heart of ABAC in Databricks
Tags are key-value metadata pairs (e.g., {"sensitivity": "confidential"}, {"region": "EMEA"}) that you can attach to any data assetโcatalogs, schemas, tables, columns, and beyond. Tags:
- Are managed and enforced at the account level via tag policies.
- Drive access control (as part of ABAC), and can be leveraged for resource usage, billing, processing, or discovery.
- Support nuanced policiesโsuch as masking โPIIโ columns for certain users or limiting data access to specific departments.
ABAC Policies and Inheritance:
- Policy Types: Four primary typesโRow Level Security (RLS), Column Level Masking (CLM), GRANT, and DENY.
- Application: Policies apply at catalog, schema, or table level and automatically inherit downward. This reduces duplication and centralizes enforcement.
- Conditions: Each policy leverages tags and other attributes to determine allowance or restriction of access, e.g., allow-only-if region = IN.
- Inheritance Model: Policies and tags can coexist with existing permission grants (RBAC); they inherit across the object hierarchy for maximum flexibility.
Fine-Grained Controls: Row Filters and Column Masks:
- Row Filter Policies: Filter table rows at query time so users only see permitted data (e.g., employees only see records from their country).
- Column Masking Policies: Apply dynamic masking functions to columns (e.g., obfuscate SSN or salary fields except for authorized users).
- Masking is decided at runtime based on the invoking userโs identity and current tags or attributes.
- Typically, these are SQL or Python UDFs, mapped to data via tags in Unity Catalog.
- Enforcement Mechanism: Once a tag is attached and a policy set, enforcement is automatic and dynamic wherever the tag applies. Modify the tag or policy and the effect cascades instantly.
Real-World Example
Suppose you have a global customer table:
- Tag the column address with { "region": "EU" } and the column ssn with { "sensitivity": "confidential" }.
- Define a row filter policy: Show only rows where region matches the userโs region attribute.
- Define a column mask: Show full SSN only for users with the โHRโ attribute, otherwise mask or hide it.
Now, a user from the EU in the Marketing department can only see EU addresses, and will never see sensitive columns like SSN.
ABAC in Action: Demo Steps
Below are practical demonstrations reflecting ABAC's capabilities, including how to set up and test row-level and column-level security using tags and policies in Databricks.
1. Creating Tags for ABAC Policies
Tags, defined as key-value pairs, help classify and govern objects. You can create it via Tag Policies :
- Row-Level Security (RLS) Tags: To control which data rows users can see.
- Column-Level Security (CLS) Tags: To mask or allow access to sensitive columns.
Example:
- RLS Tag: {"Row_level_security": "state_level_filter"}
- CLS Tag: {"column_security_tag": "ssn_masking"}
2. Creating Policies with Tags
Policies are then built to automate access control using these tags:
- Assign a policy name and description
- Specify which users/groups the policy applies to
- Select policy type: Row Filter or Column Mask
- Link to designated tags and define conditions
Example of a policy setup:
- Name: column_security
- Type: Column Mask
- Target: database or specific table
- Columns masked using UDFs depending on tag values (e.g., mask sensitive data unless the user meets certain attribute criteria)
3. Applying Tags on Datasets
Tags can be applied to data objects using three methods:
- SQL Queries
- Databricks UI
- REST API
Example:
- Assign column_security_tag.pii_ind to email and phone columns in the customer table via UI or SQL query.
- Assign row_level_security.region_level_filter to the region column in the transactions table.
4. Test Cases: Execution
Testing demonstrates how ABAC dynamically controls data access for different user personas based on policies:
- User Attribute Table: Lists persona, access rights (e.g., cost_ind = "yes"), and segment zone.
- Real Query Results: When users queries the customer table, columns with PII (email, phone) are masked based on column-level policies.
- Transaction Table Results: Only permitted rows and columns are visible according to row-level and column-level security tags and policies.
Why Adopt ABAC in Databricks?
- Centralized, auditable, and scalable: Manage security policies across all analytics and AI assets with fewer manual interventions.
- Fine-grained controls: Tightly enforce privacy and compliance mandatesโdirectly within Unity Catalogโwith less custom code.
- Less operational overhead: Tags and attribute-based rules dramatically reduce the number of roles and static permissions required.
- Adaptability: Dynamic evaluation of user, asset, and environment means greater flexibility and future-proofing for your data governance.
ABAC in Databricks is a major advance for organizations seeking next-generation data security, compliance, and operational ease across cloud-scale data platforms.
Note: ABAC is currently in Beta on AWS and Azure Databricks. Make sure to consult the documentation for the latest implementation steps and Beta conditions.
