How to stop continuous running streaming job over weekend
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-10-2023 09:43 AM
I have an continuous running streaming Job, I would like to stop this over weekend and start again on Monday. Here is my streaming job code.
(spark.readStream.format("delta").load(input_path)
.writeStream
.option("checkpointLocation", input_checkpoint_path)
.trigger(processingTime="1 minute")
.foreachBatch(foreachBatchFunction)
.start()
)
Regards
Sanjay
Labels:
- Labels:
-
Continuous Integration Pipeline