Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-25-2026 01:28 AM
Hi @Saikumar_Manne ,
Yes, you are right by saying that numPartitions would not help if the bottleneck is on the Postgres side.
AUTO CDC is very scalable solution which could easily accommodate 100M+ increments. I do think it would be faster than the current approach. Some best practices to speed up AUTO CDC:
- Set up liquid clustering on the primary keys in the target table of the AUTO CDC - this will speed up the merge operation
- Process data incrementally with auto loader or reading the table as stream
- Use serverless for better scalability
Best regards,