cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How can I activate enzyme for delta live tables (or dlt serverless) ?

nggianno
New Contributor III

Hi!

I am using Delta Live Tables and especially materialized views and i want to run a dlt pipeline but not rerun the whole view that cost time, but rather only update and add only the values that have been changed. I saw that Enzyme and does this job and dlt serverless, but I do not have this choice in my dlt pipeline. 

How can I activate that?

3 REPLIES 3

nggianno
New Contributor III

Thank you for your answer.

What I meant with my question was the following: every time that I run a delta live table materialized view, I get a log of "COMPLETE RECOMPUTE" and I saw in a Databricks official video that there is an option to instead of "COMPLETE RECOMPUTE" to have "PARTITION OVERWRITE". How can I achieve that?

devmehta
New Contributor III

I want to achieve the same as @nggianno. Please suggest a workaround

Mounika_Tarigop
Databricks Employee
Databricks Employee

To achieve "PARTITION OVERWRITE" instead of "COMPLETE RECOMPUTE" when running a Delta Live Table (DLT) materialized view, you need to configure your pipeline to use incremental refresh. This can be done by setting up your DLT pipeline to use serverless compute, which supports incremental refresh for materialized views.

https://docs.databricks.com/en/optimizations/incremental-refresh.html

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group