Databricks Job Timeout after 20 minutes

[ibrahim21124]

Hello,

I have created a job with no timeout-seconds provided. But I am getting Error: Timed out within 20 minutes. I am running the below commands using Bash@3 task in ADO Pipeline yaml file. The code for the same is given below

task: Bash@3

  timeoutInMinutes: '0'

  displayName: 'Run Databricks Tests Notebook'

  inputs:

    targetType: 'filePath'

    filePath: './ci/createjob.sh'

    arguments: '$(cluster-id) $(unit-folder) $(component-folder) $(Build.BuildId) $(Build.BinariesDirectory)'

This is the json I am passing to create a job.

databricks jobs create --json {

    "name": "Test Job",

    "timeout_seconds": 0,

    "tasks":

    [

        {

            "task_key": "abc",

            "existing_cluster_id": "cluster-id",      

            "notebook_task": {

                "notebook_path": "<path-to-notebook>",

                "source": "WORKSPACE"

                }

        }

}

Also I am starting the job with databricks run-now CLI Command 

databricks jobs run-now --json '{

  "job_id": '$JOB_ID',

  "notebook_params": {

      "param": "value"

  }

}'

Is there a way I can check where is this failing after 20 minutes?

[Kaniz_Fatma]

Hi @ibrahim21124 ,

• First, ensure that the job timeout is correctly set to 0 seconds in your job configuration.
• If you’re using the Databricks REST API to create the job, make sure the "timeout_seconds" field is set to 0.
• Double-check the job configuration to ensure there are no conflicting settings that might override the timeout.

1.  

[ibrahim21124 · Accepted solution]

Hey Fatma,

Thanks for your response. I got the solution. Actually when using Databricks CLI and triggering run-now from CLI it is having default timeout of 20 minutes. I have changed that and now it is working as expected.

Thanks,

Ibrahim