Materialize View creation not supported from DLT Pipeline
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-31-2023 10:41 AM
Hi Team, I have a very basic scenario where I am using my custom catalog and want materialize view to get created from DLT table at the end of pipeline. The SQL used as below for the same.
where "loom_data_transform" is a Streaming table. But pipeline fails at the last stage where materialize view is tried to create from "loom_data_transform" with following error message.
Could someone please help me on this issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-31-2023 05:12 PM - edited 12-31-2023 05:13 PM
Hey @abhijitnag are you sure your loom_data_transform was created as a STREAMING table?
docs:
https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-create-streaming-table.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-31-2023 06:44 PM - edited 12-31-2023 06:46 PM
Yes, @warsamebashir "loom_data_transform" was created as a STREAMING table as below. And without this Materialize view, the pipeline is executing absolutely fine.