https://community.databricks.com/t5/community-articles/why-configure-a-job-timeout/m-p/71339#M102 <P>If you use Databricks Jobs for your workloads, it is possible you might have run into a situation where you find your jobs to be in "hung" state.</P> <P>Before cancelling the job it is important to collect the thread dump as I described <A href="https://community.databricks.com/t5/community-discussions/how-to-collect-a-thread-dump-from-databricks-spark-ui/td-p/68979&nbsp;" target="_self">here</A>&nbsp;to be able to find the root cause.</P> <P>But how do we not run into a situation where our jobs are in the "hung" state for a prolonged time?</P> <P>If you know the expected time of the job completion, with a buffer you should always set Job timeouts as described <A href="https://docs.databricks.com/en/workflows/jobs/settings.html#configure-an-expected-completion-time-or-a-timeout-for-a-job" target="_self">here</A>.&nbsp;</P> <P><SPAN>To configure a maximum completion time for a job, enter the maximum duration in the&nbsp;</SPAN><STRONG>Timeout</STRONG><SPAN>&nbsp;field. If the job does not complete in this time, Databricks sets its status to “Timed Out” and the job is stopped.</SPAN></P> <P>For Streaming Jobs check my post <A href="https://community.databricks.com/t5/knowledge-sharing-hub/monitoring-a-streaming-job/m-p/71340#M103" target="_self">here</A>.</P> Sat, 01 Jun 2024 21:46:59 GMT NandiniN 2024-06-01T21:46:59Z https://community.databricks.com/t5/community-articles/why-configure-a-job-timeout/m-p/71339#M102 <P>If you use Databricks Jobs for your workloads, it is possible you might have run into a situation where you find your jobs to be in "hung" state.</P> <P>Before cancelling the job it is important to collect the thread dump as I described <A href="https://community.databricks.com/t5/community-discussions/how-to-collect-a-thread-dump-from-databricks-spark-ui/td-p/68979&nbsp;" target="_self">here</A>&nbsp;to be able to find the root cause.</P> <P>But how do we not run into a situation where our jobs are in the "hung" state for a prolonged time?</P> <P>If you know the expected time of the job completion, with a buffer you should always set Job timeouts as described <A href="https://docs.databricks.com/en/workflows/jobs/settings.html#configure-an-expected-completion-time-or-a-timeout-for-a-job" target="_self">here</A>.&nbsp;</P> <P><SPAN>To configure a maximum completion time for a job, enter the maximum duration in the&nbsp;</SPAN><STRONG>Timeout</STRONG><SPAN>&nbsp;field. If the job does not complete in this time, Databricks sets its status to “Timed Out” and the job is stopped.</SPAN></P> <P>For Streaming Jobs check my post <A href="https://community.databricks.com/t5/knowledge-sharing-hub/monitoring-a-streaming-job/m-p/71340#M103" target="_self">here</A>.</P> Sat, 01 Jun 2024 21:46:59 GMT https://community.databricks.com/t5/community-articles/why-configure-a-job-timeout/m-p/71339#M102 NandiniN 2024-06-01T21:46:59Z