cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Not able create Job via Jobs api in databricks

Pritam
New Contributor II

I am not able to create jobs via jobs API in databricks.

Error=INVALID_PARAMETER_VALUE: Job settings must be specified.

I simply copied the JSON file and saved it. Loaded the same JSON file and tried to create the job via API but the got the above error.

Another thing I tried was, copying the JSON details and assigning it to a variable in python databricks and trying to create the job, but got the same error.

Settings are there in the JSON, but some how the JSON details we are passing to create job post command are not getting in the right format.

Is there a way where I can simply load the JSON details in the databricks notebook and create the job again? I went through their documentation, but there there is no example like how they are inputting the json file in the post command.

3 REPLIES 3

Ajay-Pandey
Esteemed Contributor III

Use below format in your body section -

{ 
 
  "name": "Nightly model training", 
 
  "new_cluster": { 
 
    "spark_version": "7.3.x-scala2.12", 
 
    "node_type_id": "Standard_D3_v2", 
 
    "num_workers": 10 
 
  }, 
 
  "libraries": [ 
 
    { 
 
      "jar": "dbfs:/my-jar.jar" 
 
    }, 
 
    { 
 
      "maven": { 
 
        "coordinates": "org.jsoup:jsoup:1.7.2" 
 
      } 
 
    } 
 
  ], 
 
  "timeout_seconds": 3600, 
 
  "max_retries": 1, 
 
  "schedule": { 
 
    "quartz_cron_expression": "0 15 22 * * ?", 
 
    "timezone_id": "America/Los_Angeles" 
 
  }, 
 
  "spark_jar_task": { 
 
    "main_class_name": "com.databricks.ComputeModels" 
 
  } 
 
} 

and for header use below one-

image.png

Pritam
New Contributor II

My Header and authorization is working fine and i don't want to hard code the body of the json. I am already loading the json and also copying the json in databricks and assigning to a variable.

By the way your above code works but i don't want to hard code it. I already have some jobs and want to duplicate it in other databrick instance

rAlex
New Contributor II

@Pritam Arya​  I had the same problem today. In order to use the JSON that you can get from the GUI in an existing job, in a request to the Jobs API, you want to use just the JSON that is the value of the settings key.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.