Materialized View Refresh - NUM_JOINS_THRESHOLD_EXCEEDED?

[GregTyndall]

I have a very basic view with 3 inner joins that will only do a full refresh. Is there a limit to the number of joins you can have and still get an incremental refresh?

"incrementalization_issues": [
{
"issue_type": "INCREMENTAL_PLAN_REJECTED_BY_COST_MODEL",
"prevent_incrementalization": true,
"cost_model_rejection_subtype": "NUM_JOINS_THRESHOLD_EXCEEDED"
}
]

Thanks!

[MuthuLakshmi · Accepted solution]

@GregTyndall We have a limit of 2 joins 

[PotnuruSiva]

@GregTyndall Yes, the current limit is 2 by default. But we can increase up to 5 with the below flag added to the pipeline settings.

pipelines.enzyme.numberOfJoinsThreshold 5

[TheSmike]

I have the same issue.

What do you mean exactly with "added to the pipeline settings"? How can I set it? 

[aayrm5]

Hey @TheSmike 

In the DLT Pipeline's top right cornor, you can click on settings and scroll down to Advanced and click on Add Configuration and give the key as `pipelines.enzyme.numberOfJoinsThreshold` and value as 5.

Hope this helps.

[TheSmike]

Thanks, it works.

[_DatabricksUser]

@GregTyndall- how did you get those level of details (incrementalization_issues) for the MV build?