topic Re: Databricks Job Timeout after 20 minutes in Data Engineering

Databricks Job Timeout after 20 minutes

ibrahim21124 — Mon, 17 Jun 2024 13:54:46 GMT

Hello,

I have created a job with no timeout-seconds provided. But I am getting Error: Timed out within 20 minutes. I am running the below commands using Bash@3 task in ADO Pipeline yaml file. The code for the same is given below

task: Bash@3

timeoutInMinutes: '0'

displayName: 'Run Databricks Tests Notebook'

inputs:

targetType: 'filePath'

filePath: './ci/createjob.sh'

arguments: '$(cluster-id) $(unit-folder) $(component-folder) $(Build.BuildId) $(Build.BinariesDirectory)'

This is the json I am passing to create a job.

databricks jobs create --json {

"name": "Test Job",

"timeout_seconds": 0,

"tasks":

[

{

"task_key": "abc",

"existing_cluster_id": "cluster-id",

"notebook_task": {

"notebook_path": "<path-to-notebook>",

"source": "WORKSPACE"

}

}

Also I am starting the job with databricks run-now CLI Command

databricks jobs run-now --json '{

"job_id": '$JOB_ID',

"notebook_params": {

"param": "value"

}

Is there a way I can check where is this failing after 20 minutes?

Re: Databricks Job Timeout after 20 minutes

ibrahim21124 — Mon, 08 Jul 2024 05:57:07 GMT

Hey Fatma,

Thanks for your response. I got the solution. Actually when using Databricks CLI and triggering run-now from CLI it is having default timeout of 20 minutes. I have changed that and now it is working as expected.

Thanks,

Ibrahim

Re: Databricks Job Timeout after 20 minutes

zoeys02 — Thu, 02 Jan 2025 09:07:06 GMT

Hey Ibrahim, how did you change the default timeout? Is it the field timeout_seconds in the API config here?

Re: Databricks Job Timeout after 20 minutes

zoeys02 — Thu, 02 Jan 2025 13:11:51 GMT

Find the answer in help of jobs summit

databricks jobs submit -h

Re: Databricks Job Timeout after 20 minutes

KavyaKusuma — Fri, 16 May 2025 06:55:26 GMT

I am also facing the same issue, can you please let me know where to change the default timeout ?