โ01-07-2024 04:56 AM
Hi ,
I Want to set execution termination time/timeout limit for job in job config file. please help me how I can do this by pass some parameter in job config file.
โ01-07-2024 10:21 AM
Hi @Pratibha
You can configure optional duration thresholds for a job, including an expected completion time for the job and a maximum completion time for the job. To configure duration thresholds, click Set duration thresholds.
If you are creating jobs using JOBS JSON API then you need to set the below in JSON for timeout.
Hope this helps.
Thanks,
RK
โ01-08-2024 03:20 AM
yep got it but there is some additional requirement in my project I want if job take longer time then it will terminate and again it will try even if there is timeout error and in databricks launched status should show retry by scheduler and it should follow min_retry_interval_millis before start retry run.. how I can do this. I have set all parameter in config but once job gone in timeout it immidiatly start running and luanched status is not changing means it is not indicating my job is retried. please help
โ01-18-2024 03:03 AM
Hi @Pratibha, To set a timeout limit for a job, you can use the timeout parameter in your job configuration file. This parameter sets the maximum duration for a task. If the task does not complete within this time, the jobโs status is set to "Timed Out".
For retrying a job after a timeout, you can configure a retry policy for the task in your job configuration. This policy determines when and how many times failed task runs are retried.
However, if you want the job to retry even after a timeout error, and you want the Databricks launched status to show โretry by schedulerโ, you might need to handle this programmatically. You could write a script that checks the status of the job and if itโs โTimed Outโ, it could trigger the job again after a specified interval (min_retry_interval_millis).
If youโre using Kubernetes, you might find the activeDeadlineSeconds parameter useful. If you have further questions or need more specific advice, feel free to ask!
โ01-22-2024 03:01 AM
Thank you for posting your question in our community! We are happy to assist you.
To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?
This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!