cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

How to disable full refresh on materialized view if the src is not updated since last run

IM_01
Contributor II

yesterday

Hi,

Is there any option to control refresh rate of materialized view such as , even the dlt is triggered in full refresh mode and src Streaming tables are not updated then the full refresh should not happen on mvs . Is there any way to achieve this.

0 Kudos
1 REPLY 1

emma_s
Databricks Employee
Databricks Employee

11 hours ago

Hi,

The only way that I think you can do this is to seperate out the Materialized views from the pipelines into standalone SQL jobs with a TRIGGER ON UPDATE clause. This means they only run if the streaming tables are updated. However, this approach is fine if you just have a few MVs but if there are lots and there are many interdependencies it won't be good to manage. I think the thing to consider is that MVs only process incremental records so if the records in the source tables are changed the MV process kicks in but it will barely use any compute as it won't be doing anything.

I hope this helps.

Thanks,

Emma

0 Kudos
Announcements