Re: Databricks Job Timeout after 20 minutes

ibrahim21124 · ‎06-17-2024

Hello,

I have created a job with no timeout-seconds provided. But I am getting Error: Timed out within 20 minutes. I am running the below commands using Bash@3 task in ADO Pipeline yaml file. The code for the same is given below

task: Bash@3

timeoutInMinutes: '0'

displayName: 'Run Databricks Tests Notebook'

inputs:

targetType: 'filePath'

filePath: './ci/createjob.sh'

arguments: '$(cluster-id) $(unit-folder) $(component-folder) $(Build.BuildId) $(Build.BinariesDirectory)'

This is the json I am passing to create a job.

databricks jobs create --json {

"name": "Test Job",

"timeout_seconds": 0,

"tasks":

[

{

"task_key": "abc",

"existing_cluster_id": "cluster-id",

"notebook_task": {

"notebook_path": "<path-to-notebook>",

"source": "WORKSPACE"

}

}

Also I am starting the job with databricks run-now CLI Command

databricks jobs run-now --json '{

"job_id": '$JOB_ID',

"notebook_params": {

"param": "value"

}

}'

Is there a way I can check where is this failing after 20 minutes?

ibrahim21124 · ‎07-07-2024

Hey Fatma,

Thanks for your response. I got the solution. Actually when using Databricks CLI and triggering run-now from CLI it is having default timeout of 20 minutes. I have changed that and now it is working as expected.

Thanks,

Ibrahim

View solution in original post

zoeys02 · ‎01-02-2025

Hey Ibrahim, how did you change the default timeout? Is it the field timeout_seconds in the API config here?

zoeys02 · ‎01-02-2025

Find the answer in help of jobs summit

databricks jobs submit -h

KavyaKusuma · ‎05-15-2025

I am also facing the same issue, can you please let me know where to change the default timeout ?