Managing Row-Level Security (RLS) vs. Views: Performance impacts at scale

[Khasim_1]

 Hi everyone, we are designing our security layer in Unity Catalog. We are debating between using Row-Level Security (RLS) predicates versus standard SQL Views for masking PII data.

1. When applying complex RLS predicates to tables with >100M rows, have you noticed a significant impact on query planning time compared to traditional view-based security?
2. Are there specific patterns you use to ensure RLS is applied consistently across both SQL Warehouse queries and DLT pipelines?
3. Is there a "limit" to the complexity of the predicate functions you’d recommend before switching to a materialized view strategy?

[ThomazNeto]

Hi Khasim,

Most of this is answered directly in the Databricks docs, so let me point to what they say.

1. Planning time vs. runtime. The docs don't describe a planning-time penalty; the cost is in what the optimizer is allowed to do. Both table-level row filters/column masks and ABAC policies insert a SecureView barrier in the plan, and predicates with side effects can't be pushed across it. The performance page calls this "the most common source of performance issues with protected tables" and says it "can also block partition pruning and liquid clustering optimizations, which can force full table scans", even "when the policy UDF resolves to a constant true". Simple equality and range predicates on the protected table still push down; predicates wrapped in functions or with implicit casts don't. The page shows EXPLAIN output where the predicate moves from PartitionFilters inside the scan to a Filter above SecureView, which is a good way to check your own queries.
https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/performance

The recommendations on the overview page are short and worth following literally: use simple UDFs (prefer CASE over mapping tables or subqueries), limit the number of distinct masks on large tables, reduce UDF arguments, avoid row filters with many AND conjuncts, use deterministic expressions that can't throw (try_divide instead of /), and prefer SQL UDFs over Python. Identity functions like is_account_group_member are resolved once per query, not per row, so they're cheap; a lookup like EXISTS(SELECT 1 FROM access_table WHERE user = session_user()) is explicitly called out as slower. If you do use a mapping table, keep it small enough to broadcast, otherwise the optimizer falls back to a shuffle join. The docs also recommend testing on at least 1 million rows before production.
https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/

2. Consistency across SQL Warehouse and pipelines. Filters and masks are table-level, so enforcement is the same on a SQL warehouse, standard access mode (DBR 12.2 LTS+) and dedicated access mode (DBR 15.4 LTS+ with serverless enabled for the workspace).
https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/manually-apply

For Lakeflow pipelines, streaming tables and materialized views declare them in the table definition: WITH ROW FILTER and the MASK column clause in CREATE OR REFRESH STREAMING TABLE, or the row_filter argument plus the DDL schema string in the Python decorator. Two details from the docs matter here: during a pipeline refresh the filter and mask functions run with the pipeline owner's rights (CURRENT_USER / IS_MEMBER evaluate as the owner), and only at query time as the invoker; and pipelines can call existing UC UDFs but cannot create them, so the functions must exist before the pipeline runs. ALTER TABLE is not allowed on streaming tables, so changes go through CREATE OR REFRESH.
https://docs.databricks.com/aws/en/ldp/unity-catalog
https://docs.databricks.com/aws/en/ldp/developer/ldp-sql-ref-create-streaming-table
https://docs.databricks.com/aws/en/ldp/developer/ldp-python-ref-table

3. Complexity limit. The docs don't give a threshold, and the alternative they point to isn't a materialized view. The recommendation is: when you need the same filtering or masking across many tables, use ABAC policies rather than per-table UDFs (the docs say policy logic is evaluated more efficiently than table-specific UDFs, and users covered by an EXCEPT clause skip the UDF entirely); when you need a curated, joined or reshaped version of the data for users who don't have access to the base tables, use a dynamic view. Before committing to filters/masks, also check the limitations list: time travel does not work on tables with row filters or column masks, clones are not supported, and MERGE does not work when the policy contains nesting, aggregations, windows, limits or non-deterministic functions.
https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/
https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/abac-vs-rls-cm

Hope this helps.

[KarinDatabricks]

I advise RLS because as Tomaz said: you cannot apply time travel with masks or filters, also clones are not supported and MERGE does not work with them either.