Databricks Job Timeout after 20 minutes

ibrahim21124
Databricks Partner

‎06-17-2024 06:54 AM

Hello,

I have created a job with no timeout-seconds provided. But I am getting Error: Timed out within 20 minutes. I am running the below commands using Bash@3 task in ADO Pipeline yaml file. The code for the same is given below

task: Bash@3
  timeoutInMinutes: '0'
  displayName: 'Run Databricks Tests Notebook'
  inputs:
    targetType: 'filePath'
    filePath: './ci/createjob.sh'
    arguments: '$(cluster-id) $(unit-folder) $(component-folder) $(Build.BuildId) $(Build.BinariesDirectory)'
 

This is the json I am passing to create a job.

databricks jobs create --json {
    "name": "Test Job",
    "timeout_seconds": 0,
    "tasks":
    [
        {
            "task_key": "abc",
            "existing_cluster_id": "cluster-id",      
            "notebook_task": {
                "notebook_path": "<path-to-notebook>",
                "source": "WORKSPACE"
                }
        }
}

Also I am starting the job with databricks run-now CLI Command 
databricks jobs run-now --json '{
  "job_id": '$JOB_ID',
  "notebook_params": {
      "param": "value"
  }
}'
 
Is there a way I can check where is this failing after 20 minutes?
0 Kudos