Databricks jobs UPdate/Reset API throws an unexpected error

askme — Tue, 26 Jul 2022 20:58:50 GMT

{

"error_code": "INVALID_PARAMETER_VALUE",

"message": "Missing required field: job_id"

}

I have a test job cluster and I need to update the docker image filed with the other version using reset/update job API. I went through the documentation of data bricks and ran all of the commands that are related to the reset but I am getting the above error. attaching the snippet of my JSON file.

curl --netrc --request POST \ https://databricksinstance/api/2.0/jobs/update job_id=1 \ --data @resetjob.json \ | jq .

{

"job_id": 1,

"settings": {

"name": "testjob",

"new_cluster": {

"cluster_name": "",

"spark_version": "",

"spark_conf": {

"spark.sql.legacy.parser.havingWithoutGroupByAsWhere": "=True"

"aws_attributes": {

"zone_id": "",

"first_on_demand": 0,

"availability": "SPOT_WITH_FALLBACK",

"instance_profile_arn": "",

"spot_bid_price_percent": ,

"ebs_volume_type": "GENERAL_PURPOSE_SSD",

"ebs_volume_count": 1,

"ebs_volume_size": 100

"node_type_id": "i3.xlarge",

"driver_node_type_id": "i3.xlarge",

"custom_tags": {

"Contractor": "",

"USER": ""

"cluster_log_conf": {

"s3": {

"destination": "",

"region": "",

"enable_encryption": true,

"canned_acl": ""

}

"enable_elastic_disk": false,

"init_scripts": [

{

"dbfs": {

"destination": ""

}

{

"dbfs": {

"destination": ""

}

"docker_image": {

"url": "*",

"basic_auth": {}

"policy_id": "",

"num_workers": 12

"libraries": [

{

"egg": ""

}

"email_notifications": {

"no_alert_for_skipped_runs": false

"timeout_seconds": 0,

"notebook_task": {

"notebook_path": ""

"max_concurrent_runs": 10

"created_time": ,

"creator_user_name": "",

"run_as_user_name": ""

}

Any idea on this?

Re: Databricks jobs UPdate/Reset API throws an unexpected error

Hubert-Dudek — Wed, 27 Jul 2022 09:57:23 GMT

please don't specify job_id in url curl --netrc --request POST \ https://databricksinstance/api/2.0/jobs/update ~~job_id=1~~ \ --data @resetjob.json \ | jq .
in update there is "new_settings" not "settings"
please consider using version 2.1 instead of 2.0
please consider adding JSON header -H "Content-Type: application/json"

Re: Databricks jobs UPdate/Reset API throws an unexpected error

Vidula — Tue, 06 Sep 2022 08:58:59 GMT

Hey there @radha kilaru

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

topic Re: Databricks jobs UPdate/Reset API throws an unexpected error in Data Engineering

Databricks jobs UPdate/Reset API throws an unexpected error

Re: Databricks jobs UPdate/Reset API throws an unexpected error

Re: Databricks jobs UPdate/Reset API throws an unexpected error