cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument

Jamie_209389
New Contributor III

I am trying to call run-now with notebook_params in Azure Databricks CLI, following

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/jobs-cli

and

escapse the quotes as stated in the documentation

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/#json-string-parameters

But I got the failure messages Error: Got unexpected extra argument when calling:

databricks jobs run-now --job-id 1 --notebook-params "{\"key\":\"value\"}"

I also tried:

$runJobBody = [PSCustomObject]@{

      key="vaule "

      placeholder="placeHolder"

 }

$runJobBody = $runJobBody | ConvertTo-Json -Compress

$runJobBody | out-file -encoding ASCII run-job.json

 databricks jobs run-now --job-id 1 --notebook-params run-job.json

But it gave the same error message.

Do anyone know how I should pass the parameters to databricks jobs run-now?

The version of databricks CLI got used is databricks-cli-0.17.x

1 ACCEPTED SOLUTION

Accepted Solutions

Also tried with version set to 2.1, still not working:

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params '{"key", "value"}'

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{"key", "value"}"

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{\"key\", \"value\"}"

Error: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Error: Got unexpected extra argument (key, value})

Error: Got unexpected extra argument (key\, \value\})

But the same json string works with Databricks API calls, so I will use the API call directly.

View solution in original post

7 REPLIES 7

Anonymous
Not applicable

@Jamie Iong​ : It looks like the issue you're encountering is due to how the quotes are being escaped. In the command you provided, the backslashes before the quotes are getting interpreted as part of the argument rather than as an escape character. Try using single quotes around the JSON string and double quotes inside the JSON string to encapsulate the key-value pairs. For example:

databricks jobs run-now --job-id 1 --notebook-params '{"key":"value"}'

Jamie_209389
New Contributor III

I tried with single quote wrapping with double quote inside:

databricks jobs run-now --job-id 1 --notebook-params '{"key", "value"}' 

databricks jobs run-now --job-id 1 --notebook-params "{"key", "value"}"

databricks jobs run-now --job-id 1--notebook-params "{\"key\", \"value\"}"

But I got these corresponding errors:

Error: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Error: Got unexpected extra argument (key, value})

Error: Got unexpected extra argument (key\, \value\})

Also tried with version set to 2.1, still not working:

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params '{"key", "value"}'

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{"key", "value"}"

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{\"key\", \"value\"}"

Error: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Error: Got unexpected extra argument (key, value})

Error: Got unexpected extra argument (key\, \value\})

But the same json string works with Databricks API calls, so I will use the API call directly.

DBXC
Contributor

Facing the same issue.

Could someone provide an update on the CLI side using run-now command with JSON notebook parameters? Is this a bug within the CLI? 

DBXC
Contributor

Update / Solved: 

Using CLI on Linux/MacOS: 

Send in the sample json with job_id in it. 
databricks jobs run-now --json '{  
 "job_id":<job-ID>,
  "notebook_params": {
    <key>:<value>,
    <key>:<value>
  }
}'

 

Using CLI on Windows: 

Send in the sample json with job_id in it and sringify the json as below
databricks jobs run-now --json '{ \"job_id\":<job-ID>, \"notebook_params\": { <key>:<value>, <key>:<value> } }'

Vaitheesh
New Contributor II

I have the latest Databricks CLI setup and configured in my Ubuntu VM. When I tried to run a job using the json template I generated using databricks jobs get 'xxxjob_idxxx' > orig.json it throws an unknown error.

Databricks CLI v0.216.0

databricks jobs run-now --json 'orig.json'

Error: invalid character 'o' looking for beginning of value

databricks jobs run-now --json file.json

Error: invalid character 'i' in literal false (expecting 'a')

I have to update the notebook parameters using 'jq' and trigger the job via Databricks CLI and this is blocking me to move forward.

Vaitheesh
New Contributor II

Found the solution!

I just need to add '@' before the json file I'm inputting like this.

databricks jobs run-now --json @file.json

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group