- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thursday
I’m trying to create a triggered Lakebase synced table from a SDP-created materialized view.
The source MV uses Automatic CDF: row tracking is enabled and legacy CDF (delta.enableChangeDataFeed) is disabled. The Automatic CDF workspace preview is enabled (says GA now)
The producer pipeline is configured with the PREVIEW channel and pipelines.externalMetadata.enabled=true. I recreated and reran the pipeline successfully.
However, synced-table creation still says that Change Data Feed is required and suggests running ALTER TABLE ... SET TBLPROPERTIES (delta.enableChangeDataFeed=true). That would enable legacy CDF, and ALTER TABLE also cannot be used because the source is a materialized view.
Does Lakebase synced-table creation support Automatic CDF for pipeline-created materialized views? If so, how should pipeline_channel=PREVIEW be configured for the managed sync pipeline?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thursday
Short answer: No — Lakebase synced tables in Triggered/Continuous mode do not currently support Automatic CDF from materialized views. This is a documented limitation, not a configuration issue.
The synced tables documentation explicitly states:
"Sources that do not support CDF (such as views, materialized views, and Iceberg tables) can only be synced in Snapshot mode."
Why it doesn't work despite your correct setup:
Your producer pipeline configuration is correct for general CDF consumption (PREVIEW channel + pipelines.externalMetadata.enabled=true + row tracking). This setup does allow reading CDF from a materialized view via table_changes() or STREAM ... WITH (readChangeFeed=true) in your own custom pipelines.
However, the managed sync pipeline that powers Lakebase synced tables only checks for legacy CDF (delta.enableChangeDataFeed=true) — it does not (yet) consume Automatic CDF. Since you can't ALTER TABLE ... SET TBLPROPERTIES on a materialized view, you hit a dead end.
Additionally, reading CDF from a materialized view is still in Beta — so it's not surprising the managed sync pipeline hasn't integrated it yet.
Workarounds:
Use Snapshot mode — works with materialized views today, no CDF required. Trade-off: full re-copy each sync, no incremental updates.
Replace the MV with a streaming table as your sync source — streaming tables support legacy CDF and work with Triggered/Continuous synced tables. If your MV logic is simple enough, you may be able to express it as a streaming table with the same transformations.
Add an intermediate Delta table — have your pipeline write results into a regular Delta table (with delta.enableChangeDataFeed=true), then point the synced table at that instead of the MV.
File a feature request — this is a reasonable ask. The building blocks exist (Automatic CDF from MVs + synced table triggered mode), they just aren't wired together yet in the managed sync pipeline.
Setting pipelines.channel=PREVIEW on the managed sync pipeline won't help here — that config controls the producer pipeline's channel, but the sync pipeline is a separate managed pipeline that you don't directly configure at that level.
