Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-03-2022 05:31 AM
If you need checkpointing, please try the below code. Thanks to persist, you will avoid reprocessing:
df = ps.sql(sql).persist()
df.spark.checkpoint()My blog: https://databrickster.medium.com/
