cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

Can we able to create materialized view in databricks using all purpose cluster

Shivaprasad
Contributor

2 hours ago

I was unable to create materialized view in databricks using all purpose cluster wanted to check do we need serverless cluster to create MV

0 Kudos
1 REPLY 1

szymon_dybczak
Esteemed Contributor III

36m ago - last edited 28m ago

Hi @Shivaprasad ,

Nope, all purpose cluster are not supported. Standalone materialized views can be created/refreshed either from a Unity Catalog-enabled Pro or Serverless SQL Warehouse, or from a notebook attached to Serverless General Compute. 

So we donโ€™t necessarily need a โ€œserverless clusterโ€ specifically, but we do need one of the supported compute options - I.e a SQL Warehouse. An all-purpose cluster is not supported for creating standalone materialized views.

For materialized view created via pipeline you can choose serverless or classic compute: 

https://docs.databricks.com/aws/en/ldp/configure-compute

Here is similar thread:

https://community.databricks.com/t5/data-engineering/spark-declarative-pipelines-use-in-all-purpose-...

If my answer was helpful, please consider marking it as accepted solution

0 Kudos
Announcements