Want to set execution termination time/timeout limit for job in job config

[Pratibha]

Hi , 

I Want to set execution termination time/timeout limit for job in job config file. please help me how I can do this by pass some parameter in job config file. 

[RKNutalapati]

Hi @Pratibha 

You can configure optional duration thresholds for a job, including an expected completion time for the job and a maximum completion time for the job. To configure duration thresholds, click Set duration thresholds.

If you are creating jobs using JOBS JSON API then you need to set the below in JSON for timeout.

Hope this helps.

Thanks,

RK

[Pratibha]

yep got it but there is some additional requirement in my project I want if job take longer time then it will terminate and again it will try even if there is timeout error and in databricks launched status should show retry by scheduler and it should follow min_retry_interval_millis before  start retry run.. how I can do this. I have set all parameter in config but once job gone in timeout it immidiatly start running and luanched status is not changing means it is not indicating my job is retried. please help

[Kaniz]

Hi @Pratibha, To set a timeout limit for a job, you can use the timeout parameter in your job configuration file. This parameter sets the maximum duration for a task. If the task does not complete within this time, the job’s status is set to "Timed Out".

For retrying a job after a timeout, you can configure a retry policy for the task in your job configuration. This policy determines when and how many times failed task runs are retried.

However, if you want the job to retry even after a timeout error, and you want the Databricks launched status to show “retry by scheduler”, you might need to handle this programmatically. You could write a script that checks the status of the job and if it’s “Timed Out”, it could trigger the job again after a specified interval (min_retry_interval_millis).

If you’re using Kubernetes, you might find the activeDeadlineSeconds parameter useful. If you have further questions or need more specific advice, feel free to ask!

[Kaniz]

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!