Can I have sequence guarantee when replicate with CDF
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2024 12:29 AM
Hi team,
I have a delta table src, and somehow I want to replicate it to another table tgt with CDF, sort of
(spark
.readStream
.format("delta")
.option("readChangeFeed", "true")
.table('src')
.writeStream
.format("delta")
.outputMode("append")
.option("checkpointLocation", 'xxx')
.toTable('tgt'))
Thanks to CDF, in tgt table I can have _commit_version. Can I have guarantee the _commit_version shows in the right sequence as they are in src table?
Thanks