Do you can schedule this notebook as separate Job on dedicated job cluster that start everyday at 8pm.I think that using Serverless cluster for this task will be the best way.
You can terminate any Job with REST API.I recommend to use Python SDK jobs.cancel_all_runs() methodhttps://databricks-sdk-py.readthedocs.io/en/latest/workspace/jobs/jobs.html#databricks.sdk.service.jobs.JobsExt.cancel_all_runs
