Materialized Views Without DLT?

[ChristianRRL]

I'm curious, is DLT *required* to use Materialized Views in Databricks? Is it not possible to create and refresh a Materialized view via a standard Databricks Workflow?

[raphaelblg]

Hi @ChristianRRL,

There are DLT materialized views: https://docs.databricks.com/en/delta-live-tables/index.html#materialized-view

and

Databricks SQL Materialized Views: https://docs.databricks.com/en/sql/user/materialized-views.html#use-materialized-views-in-databricks....

[ChristianRRL]

Hi @raphaelblg , sorry but I think you misunderstood my question. I'm aware that I can create a DLT pipeline from scratch to create Materialized Views, but I was surprised when I was attempting to create a Materialized View without trying to use DLT, but when I ran this in a standard notebook (connected to our configured cluster) I see that it does seem require DLT:

My question is, either via a Python or SQL notebook cell, can I (1) create, and (2) refresh a Materialized View without requiring DLT at all???

[ChristianRRL]

For additional clarity, I would like to show that when I attempt to run this in the SQL Editor with the SQL Starter Warehouse, I get an error because we do not have UC enabled. But I'm wanting to get the Materialized View functionality without fully requiring DLT or UC:

[raphaelblg]

@ChristianRRL, You either need a DLT pipeline or a SQL Warehouse with UC access.

"In Databricks SQL, materialized views are Unity Catalog managed tables that allow users to precompute results based on the latest version of data in source tables. ".

At: https://docs.databricks.com/en/sql/user/materialized-views.html#what-are-materialized-views

[gbattawk]

Hi @raphaelblg ,
I am also not able to create mview through UC enabled SQL Warehouse (Pro). I get error: Failed to start the DLT service on cluster xxx-xxxx-xxxxxx-xxx. An Azure storage request was not authorized. The storage account's 'Firewalls and virtual networks' settings may be blocking access to storage services. Validate your Azure storage credentials or firewall exception settings. Why does the create MV functionality need serverless compute if I want to stay with Pro compute.

[hari-prasad]

Hi @ChristianRRL ,

When creating a materialized view in Databricks, the data is stored in DBFS, cloud storage, or Unity Catalog volume. You can still create a materialized view by overwriting the same table each time, instead of using Append, Update, or Delete operations. Be sure to manage the delta table retention policy and perform vacuuming for optimal performance.

Regards,
Hari Prasad