- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thursday - last edited Thursday
Hello,
I have defined Lakebase Change Data Feed (CDF) in order to sync some lakebase native tables into my catalog.schema, i can see that the tables are created with a naming convention like this: lb_*_history.
sometimes the status is persisting in snapshoting for hours and when it is on Enabled status, the insert of a new row in lakebase side will not sync with the history tables in UC side. (Ony the first version was snapshot)
Is there any way to solve this issue?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thursday
Thanks for sharing the screenshots. Assuming this is Lakebase Change Data Feed (formerly Lakehouse Sync) from Lakebase Postgres into Unity Catalog lb_*_history tables, this does not look like expected steady-state behavior.
A few things stand out:
-
One table is in
Enabledstate, but itsLast updateis 2 days old. That usually suggests the feed is not advancing even though it is not marked as failed. -
Other tables are still in
Snapshotting. That can be normal during the initial load, especially for larger tables, but if a table has been stuck there for more than ~24 hours with no visible progress, I’d treat that as a red flag. -
One table is in
Error, which generally indicates a real feed issue rather than normal CDF lag.
A few checks I’d recommend:
-
Confirm the source tables are native Lakebase tables in the
databricks_postgresdatabase. -
Verify the affected tables have
REPLICA IDENTITY FULLset, since that is required for Lakebase CDF. -
In the Lakebase SQL editor, run:
This should show the per-table `status`, `committed_lsn`, and `last_write_time`, and is usually the best way to confirm whether the feed is actually progressing.SELECT * FROM wal2delta.tables; -
For the table in
Error, check any error details exposed in the Lakebase CDF UI right away. If the UI surfaces an error message, that will usually be the fastest clue to whether this is a permissions issue, schema issue, or something else operational. -
Check whether there were any recent schema changes on those tables. Schema changes can trigger a full re-snapshot, and because the feature is still in Public Preview, recent column adds/drops/type changes are especially worth calling out when troubleshooting.
If the table remains in Enabled but last_write_time does not move after new inserts, if Snapshotting continues beyond a reasonable initial-load window, or if a table stays in Error, I’d recommend opening a Databricks support ticket and including:
-
the CDF status screenshots
-
output of
SELECT * FROM wal2delta.tables; -
confirmation that
REPLICA IDENTITY FULLis set -
whether the tables recently had schema changes or are especially large
-
any exact error text shown in the UI
One final note: this diagnostic path applies specifically to Lakebase CDF / Lakebase Postgres → Unity Catalog history tables. If the workflow is actually Unity Catalog → Lakebase synced tables, the troubleshooting path is different.
So in short: Snapshotting can be normal during the initial load, but Enabled with no updates for days, Snapshotting with no progress for an extended period, and especially Error are all signs that the feed likely needs deeper investigation.