cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

API error when creating serverless job "MALFORMED_REQUEST"

p_hcarvalho
New Contributor II

I tried executing the job run summit API (/api/2.1/jobs/runs/submit)

I have two executions of this API, the first one is working, but the second one don't,

I'm getting the following error:

{"error_code":"MALFORMED_REQUEST","message":"Invalid JSON given in the body of the request - failed to parse given JSON"}


Can anyone give me some advice?

First JSON that is working:

{
"run_name":"my-run-name",
"tasks":[
{
"task_key":"my-task-name",
"notebook_task":{
"notebook_path":"my/notebook/path",
"source":"WORKSPACE",
"base_parameters":{
"fileName":"sampledummie.xlsx",
"pathInputLanding":"/path/to/my/file/sampledummie0/",
"pathOutputLanding":"/path/to/my/output/file/sampledummie0/",
"executionUUID":"d171a035-fe97-46c3-873f-14d91bede7e1"
}
},
"timeout_seconds":36000
}
],
"webhook_notifications":{
"on_start":[
{
"id":"webhook_id_hash"
}
],
"on_failure":[
{
"id":"webhook_id_hash"
}
],
"on_success":[
{
"id":"webhook_id_hash"
}
]
},
"access_control_list":[
{
"group_name":"users",
"permission_level":"CAN_VIEW"
}
]
}


Second JSON that is not working

{
"run_name":"my-other-run-name",
"tasks":[
{
"task_key":"my-other-task-name",
"notebook_task":{
"notebook_path":"my/notebook/path",
"source":"WORKSPACE",
"base_parameters":{
"executionUUID":"d171a035-fe97-46c3-873f-14d91bede7e1",
"pathOutputLanding":"/path/to/my/output/file/sampledummie0/",
"notebooks":"{ 'notebooks':[ 'path/to/notebook1', 'path/to/notebook2', 'path/to/notebook3', 'path/to/notebook3', 'path/to/notebook4', 'path/to/notebook5', 'path/to/notebook6', 'path/to/notebook7' ]}",
"files":"{'files':[{'file_name':'sampledummie_base.xlsx','sheets':[{'sheet_name':'AAAA','gold':'table_name1'},{'sheet_name':'BBBB','gold':'table_name2'},{'sheet_name':'CCCC','gold':'table_name3'},{'sheet_name':'DDDD','gold':'table_name4'}]},{'file_name':'sampledummie_parameters.xlsx','sheets':[{'sheet_name':'AAAA','gold':'table_name1'},{'sheet_name':'BBBB','gold':'table_name2'},{'sheet_name':'CCCC','gold':'table_name2'},{'sheet_name':'DDDD','gold':'table_name3'}]}]}",
"notebookSilver":"path/to/my/notebook"
}
},
"timeout_seconds":36000
}
],
"webhook_notifications":{
"on_start":[
{
"id":"webhook_id_hash"
}
],
"on_failure":[
{
"id":"webhook_id_hash"
}
],
"on_success":[
{
"id":"webhook_id_hash"
}
]
},
"access_control_list":[
{
"group_name":"users",
"permission_level":"CAN_VIEW"
}
]
}

The notebooks and files parameters need to be strings because that is how my notebook process them.


 

2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

Hi @p_hcarvalho,

Thanks for your question, here's a corrected version of your JSON payload, can you try it?

{
"run_name": "my-other-run-name",
"tasks": [
{
"task_key": "my-other-task-name",
"notebook_task": {
"notebook_path": "my/notebook/path",
"source": "WORKSPACE",
"base_parameters": {
"executionUUID": "d171a035-fe97-46c3-873f-14d91bede7e1",
"pathOutputLanding": "/path/to/my/output/file/sampledummie0/",
"notebooks": "{\"notebooks\": [\"path/to/notebook1\", \"path/to/notebook2\", \"path/to/notebook3\", \"path/to/notebook4\", \"path/to/notebook5\", \"path/to/notebook6\", \"path/to/notebook7\"]}",
"files": "{\"files\": [{\"file_name\": \"sampledummie_base.xlsx\", \"sheets\": [{\"sheet_name\": \"AAAA\", \"gold\": \"table_name1\"}, {\"sheet_name\": \"BBBB\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"CCCC\", \"gold\": \"table_name3\"}, {\"sheet_name\": \"DDDD\", \"gold\": \"table_name4\"}]}, {\"file_name\": \"sampledummie_parameters.xlsx\", \"sheets\": [{\"sheet_name\": \"AAAA\", \"gold\": \"table_name1\"}, {\"sheet_name\": \"BBBB\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"CCCC\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"DDDD\", \"gold\": \"table_name3\"}]}]}",
"notebookSilver": "path/to/my/notebook"
}
},
"timeout_seconds": 36000
}
],
"webhook_notifications": {
"on_start": [
{
"id": "webhook_id_hash"
}
],
"on_failure": [
{
"id": "webhook_id_hash"
}
],
"on_success": [
{
"id": "webhook_id_hash"
}
]
},
"access_control_list": [
{
"group_name": "users",
"permission_level": "CAN_VIEW"
}
]
}

I have tried that solution but it didn't work, any other suggestion?

I'm running it in Azure Data Factory also

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group