Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-11-2026 02:41 PM
So you are saying that first set the .option("startingOffsets", "latest") to read all the latest records and then delete the old checkpoint path and create the new one for it and do you recommend to use the checkpoint like below so it dont happen in the future.
# Generate a unique suffix using timestamp and UUID
unique_suffix = datetime.now().strftime("%Y%m%d_%H%M%S") + "_" + str(uuid.uuid4())
# Combine to form a unique checkpoint path
checkpoint_path = f"{base_checkpoint_dir}/checkpoint_{unique_suffix}"
