topic Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument in Data Engineering

In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument

Jamie_209389 — Mon, 13 Mar 2023 22:24:03 GMT

I am trying to call run-now with notebook_params in Azure Databricks CLI, following

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/jobs-cli

and

escapse the quotes as stated in the documentation

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/#json-string-parameters

But I got the failure messages Error: Got unexpected extra argument when calling:

databricks jobs run-now --job-id 1 --notebook-params "{\"key\":\"value\"}"

I also tried:

$runJobBody = [PSCustomObject]@{

key="vaule "

placeholder="placeHolder"

}

$runJobBody = $runJobBody | ConvertTo-Json -Compress

$runJobBody | out-file -encoding ASCII run-job.json

databricks jobs run-now --job-id 1 --notebook-params run-job.json

But it gave the same error message.

Do anyone know how I should pass the parameters to databricks jobs run-now?

The version of databricks CLI got used is databricks-cli-0.17.x

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument

Anonymous — Tue, 14 Mar 2023 08:44:52 GMT

@Jamie Iong : It looks like the issue you're encountering is due to how the quotes are being escaped. In the command you provided, the backslashes before the quotes are getting interpreted as part of the argument rather than as an escape character. Try using single quotes around the JSON string and double quotes inside the JSON string to encapsulate the key-value pairs. For example:

databricks jobs run-now --job-id 1 --notebook-params '{"key":"value"}'

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument

Jamie_209389 — Tue, 14 Mar 2023 17:06:49 GMT

I tried with single quote wrapping with double quote inside:

databricks jobs run-now --job-id 1 --notebook-params '{"key", "value"}'

databricks jobs run-now --job-id 1 --notebook-params "{"key", "value"}"

databricks jobs run-now --job-id 1--notebook-params "{\"key\", \"value\"}"

But I got these corresponding errors:

Error: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Error: Got unexpected extra argument (key, value})

Error: Got unexpected extra argument (key\, \value\})

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected extra argument

Jamie_209389 — Tue, 14 Mar 2023 18:08:00 GMT

Also tried with version set to 2.1, still not working:

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params '{"key", "value"}'

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{"key", "value"}"

databricks jobs run-now --version 2.1 --job-id 1 --notebook-params "{\"key\", \"value\"}"

Error: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Error: Got unexpected extra argument (key, value})

Error: Got unexpected extra argument (key\, \value\})

But the same json string works with Databricks API calls, so I will use the API call directly.

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected ext

DBXC — Mon, 28 Aug 2023 16:15:14 GMT

Facing the same issue.

Could someone provide an update on the CLI side using run-now command with JSON notebook parameters? Is this a bug within the CLI?

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected ext

DBXC — Tue, 29 Aug 2023 21:10:52 GMT

Update / Solved:

Using CLI on Linux/MacOS:

Send in the sample json with job_id in it.
databricks jobs run-now --json '{
"job_id":<job-ID>,
"notebook_params": {
<key>:<value>,
<key>:<value>
}
}'

Using CLI on Windows:

Send in the sample json with job_id in it and sringify the json as below
databricks jobs run-now --json '{ \"job_id\":<job-ID>, \"notebook_params\": { <key>:<value>, <key>:<value> } }'

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected ext

Vaitheesh — Thu, 04 Apr 2024 06:39:49 GMT

I have the latest Databricks CLI setup and configured in my Ubuntu VM. When I tried to run a job using the json template I generated using databricks jobs get 'xxxjob_idxxx' > orig.json it throws an unknown error.

Databricks CLI v0.216.0

databricks jobs run-now --json 'orig.json'

Error: invalid character 'o' looking for beginning of value

databricks jobs run-now --json file.json

Error: invalid character 'i' in literal false (expecting 'a')

I have to update the notebook parameters using 'jq' and trigger the job via Databricks CLI and this is blocking me to move forward.

Re: In Azure Databricks CLI, how to pass in the parameter notebook_params? Error: Got unexpected ext

Vaitheesh — Thu, 04 Apr 2024 06:48:00 GMT

Found the solution!

I just need to add '@' before the json file I'm inputting like this.

databricks jobs run-now --json @file.json