- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-24-2024 01:18 PM
Hello @Splush,
There are 2 ways to create materialized views at the current moment:
1. Through Databricks SQL: Use Materialized Views in Databricks SQL. These are the limitations.
2. Through DLT: Materialized View (DLT). All DLT tables are subject to these limitations.
The reason why the statements fail on interactive or job clusters is because materialized view declaration is only supported on DBSQL or DLT, there's no random reason.
Now regarding RLS (Row Level Security), these are the docs: Filter sensitive table data using row filters and column masks. And these are the Limitations.
Regarding your question:
Am I misunderstanding something here maybe?
Our goal would be to have the Query as CREATE OR REFRESH and WITH ROW FILTER running inside of a DLT pipeline.
At the current stage of the preview (2024-07-24), one of the limitations is: "Materialized views and streaming tables declared in Delta Live Tables don’t support row filters or column masks.". I believe this statement should answer your question.
Databricks is working on improving the DLT compatibility with overall UC features and you can expect to see this limitation being removed on future DLT releases.
Raphael Balogo
Sr. Technical Solutions Engineer
Databricks