Today I ran into an interesting issue while setting up a Lakebase synced table to sync a Delta table from our Lakehouse to Postgres using Continuous Sync mode.
Everything looked correct:
- Change Data Feed (CDF) was enabled.
- The pipeline was healthy.
- The sync was running successfully.
Yet the destination Postgres table remained completely empty.
There were no errors or warnings, which made the issue harder to diagnose.
After investigating, I found that the problem was the primary key.
I had configured CONTRACT_NO as the primary key because it seemed like the obvious business identifier. However, the source was a history table, so multiple rows existed for the same contract.
Since Continuous Sync relies on a unique primary key to determine INSERT, UPDATE, and DELETE operations, duplicate values meant the sync couldn't uniquely identify records.
The solution was to:
- Use a composite primary key (CONTRACT_NO, REPORT_DATE)
- Deduplicate the source data before syncing
One quick query immediately exposed the issue:
SELECT CONTRACT_NO, COUNT (*) FROM source table GROUP BY CONTRACT_NO HAVING COUNT (*) > 1;
Key takeaway: Before assuming there's an issue with the pipeline or platform, validate your data model. A business identifier isn't always a true primary keyâespecially in history or snapshot tables.
Has anyone else encountered similar "no data, no error" situations with Lake base Continuous Sync? I'd love to hear what caused them and how you resolved them.
As
