cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Sync Tables: Unity Catalog to Lakebase - Materialized Views triggered mode

Pat
Esteemed Contributor

Hi,

I am facing some issues syncing Materialized Views into Lakebase with triggered mode, and I actually wonder if this is actually possible.

create materialized view some_catalog.some_schema.some_table
tblproperties(
  'delta.enableChangeDataFeed' = 'true',
  'pipelines.channel'='PREVIEW')
as
select * from some_catalog_src.some_schema_src.some_table_src limit 100;

I've already added params that are needed, you can see below

Pat_0-1780047815206.png

when trying to create synced table, I get following error:

 

Incremental sync for table some_catalog_src.some_schema_src.some_table_src requires Auto CDF to identify changes, which is only available in the PREVIEW channel. Set pipeline_channel = PREVIEW to proceed.

This materialized view is created manually, but I have also tested on MV created via SDP (in PREVIEW channel).

this works with tables:

create table some_catalog.some_schema.some_table_tbl
tblproperties(
  'delta.enableChangeDataFeed' = 'true')
as
select * from some_catalog_src.some_schema_src.some_table_tbl_src limit 100;

 The source for triggered sync needs to have CDF enabled:

Pat_2-1780048797934.png

and I believe I have this enabled, but just noticed it's in Private Preview, so will need to double check:

https://www.reddit.com/r/databricks/comments/1rl264r/materialized_view_change_data_feed_cdf_private/

while I wrote this, I will leave it and double check if I have enabled CDF on MVs

0 REPLIES 0