topic Re: Incremental refresh - non serverless compute in Data Engineering

Incremental refresh - non serverless compute

RotemBar — Tue, 31 Dec 2024 14:24:46 GMT

Hey,

I read the page about incremental refresh. Will you make it available on more than just serverless compute?

If so, when?

Thanks

Reference - https://docs.databricks.com/en/optimizations/incremental-refresh.html

Re: Incremental refresh - non serverless compute

Alberto_Umana — Tue, 31 Dec 2024 14:28:11 GMT

Hi @RotemBar,

The documentation specifies that incremental refreshes are supported only when using serverless pipelines, whether in Databricks SQL or Delta Live Tables (DLT) pipelines configured to use serverless compute.

There is no information available about plans or timelines for making incremental refresh available on non-serverless compute. I can check internally if there are plans in near future.

Re: Incremental refresh - non serverless compute

RotemBar — Tue, 31 Dec 2024 14:30:26 GMT

I'd really appreciate it

Re: Incremental refresh - non serverless compute

Alberto_Umana — Tue, 31 Dec 2024 14:44:21 GMT

Sure thing, will keep you posted in a DM.