Run continuous job for a period of time

[theanhdo]

Hi there,

I have a job where the Trigger type is configured as Continuous. I want to only run the Continuous job for a period of time per day, e.g. 8AM - 5PM. I understand that we can achieve it by manually starting and cancelling the job on the UI, or by programmatically starting and cancelling the job using these APIs
https://<databricks-instance>/api/2.1/jobs/run-now
https://<databricks-instance>/api/2.1/jobs/runs/cancel

However, I would like to ask if there is any job setting, e.g. using cron syntax, to achieve this?

[MuthuLakshmi]

@theanhdo 

To schedule a job to run at 8 AM every day, you should use the Scheduled trigger type rather than the Continuous trigger type. The Continuous trigger type is designed to keep a job running continuously, which is not suitable for running a job at a specific time each day.

Here’s how you can schedule a job to run at 8 AM every day using the Scheduled trigger type:

1. Navigate to Workflows:
• In the Databricks workspace, go to the sidebar and click on Workflows.
2. Select the Job:
• Click the job name in the Name column on the Jobs tab.
3. Add a Trigger:
• In the Job details panel, click Add trigger.
4. Configure the Trigger:
• In the Trigger type dropdown, select Scheduled.
• In the Schedule type dropdown, select Advanced.
5. Set the Schedule:

Use the following cron expression to schedule the job to run at 8 AM every day:

0 8 * * *

Optionally, select the Show Cron Syntax checkbox to display and edit the schedule using Quartz Cron Syntax.

1. Save the Configuration:
• Click Save to apply the schedule.

This configuration will ensure that your job runs at 8 AM every day.

To stop the job, you have to use rest API

You will need to create a separate job or task that stops the main job at 5 PM. This can be done using the Databricks REST API to cancel the job run.

Create a new job that uses the REST API to cancel the main job run.

Add a trigger to this new job with the following cron expression to run at 5 PM every da

[theanhdo]

Hi @MuthuLakshmi , thank you for your answer. However, your answer doesn't help with my question. Let me rephrase my question.

In short, my question is how to configure a Continuous job to run for a period of time, e.g. from 8AM to 5PM every day, and automatically stop in other time of the day?

In details, I have a job that is running Continuously from 8AM to 5PM every day, and in other time of the day I want to stop it. The job is configured with the Trigger set as Continuous, however there is no option to configure the running period. I understand that we can achieve it by manually starting and cancelling the job on the UI, or by programmatically starting and cancelling the job using these APIs
https://<databricks-instance>/api/2.1/jobs/run-now
https://<databricks-instance>/api/2.1/jobs/runs/cancel

However, I would like to ask if there is any job setting, e.g. using cron syntax, to achieve this?