Evaluating View-Based Access Control vs. Native Databricks Security Features

jackgurae — Tue, 16 Jul 2024 05:47:26 GMT

Hello Databricks Community,

I'm seeking advice and insights on best practices for managing data access and permissions in Databricks. Our company currently uses a view-based approach for access control, but I'm wondering if we should transition to Databricks' native security features like row-level security (RLS) and column masks.

Our Current Approach
We create multiple view tables from upstream tables to manage permissions. For example:
- `client_info` (upstream table)
- `npii_client_info`
- `pii_client_info`
- `pii_client_info_teamA`

Concerns
1. Data lineage complexity??
2. Maintenance overhead (multiple data dictionaries)
3. Potential inconsistencies across views
4. Scalability as data and organization grow

1. What are the pros and cons of our view-based approach vs. using Databricks' RLS and column masks?
2. Has anyone successfully transitioned from a view-based system to native Databricks security features? What challenges did you face?
3. Are there specific use cases where view-based access control might be preferable?
4. How does the performance compare between these two approaches, especially for large datasets?
5. What impact does each approach have on data governance and compliance efforts?

I'd greatly appreciate any insights, experiences, or best practices you can share. Thank you in advance for your help!

Re: Evaluating View-Based Access Control vs. Native Databricks Security Features

jackgurae — Wed, 24 Jul 2024 09:17:49 GMT

Thank you @Retired_mod for your contribution.

I want to learn more about real use cases when company decided to go with Databricks' Row-Level Security (RLS) and Column Masks. I just want to convince teams to consider the approach.
To me the pros outweigh the cons, but i want to know what angle I might miss.

topic Re: Evaluating View-Based Access Control vs. Native Databricks Security Features in Data Governance

Evaluating View-Based Access Control vs. Native Databricks Security Features

Re: Evaluating View-Based Access Control vs. Native Databricks Security Features