topic Re: API error when creating serverless job "MALFORMED_REQUEST" in Data Engineering

API error when creating serverless job "MALFORMED_REQUEST"

p_hcarvalho — Thu, 23 Jan 2025 13:17:40 GMT

I tried executing the job run summit API (/api/2.1/jobs/runs/submit)

I have two executions of this API, the first one is working, but the second one don't,

I'm getting the following error:

{"error_code":"MALFORMED_REQUEST","message":"Invalid JSON given in the body of the request - failed to parse given JSON"}

Can anyone give me some advice?

First JSON that is working:

{
"run_name":"my-run-name",
"tasks":[
{
"task_key":"my-task-name",
"notebook_task":{
"notebook_path":"my/notebook/path",
"source":"WORKSPACE",
"base_parameters":{
"fileName":"sampledummie.xlsx",
"pathInputLanding":"/path/to/my/file/sampledummie0/",
"pathOutputLanding":"/path/to/my/output/file/sampledummie0/",
"executionUUID":"d171a035-fe97-46c3-873f-14d91bede7e1"
}
},
"timeout_seconds":36000
}
],
"webhook_notifications":{
"on_start":[
{
"id":"webhook_id_hash"
}
],
"on_failure":[
{
"id":"webhook_id_hash"
}
],
"on_success":[
{
"id":"webhook_id_hash"
}
]
},
"access_control_list":[
{
"group_name":"users",
"permission_level":"CAN_VIEW"
}
]
}

Second JSON that is not working

{
"run_name":"my-other-run-name",
"tasks":[
{
"task_key":"my-other-task-name",
"notebook_task":{
"notebook_path":"my/notebook/path",
"source":"WORKSPACE",
"base_parameters":{
"executionUUID":"d171a035-fe97-46c3-873f-14d91bede7e1",
"pathOutputLanding":"/path/to/my/output/file/sampledummie0/",
"notebooks":"{ 'notebooks':[ 'path/to/notebook1', 'path/to/notebook2', 'path/to/notebook3', 'path/to/notebook3', 'path/to/notebook4', 'path/to/notebook5', 'path/to/notebook6', 'path/to/notebook7' ]}",
"files":"{'files':[{'file_name':'sampledummie_base.xlsx','sheets':[{'sheet_name':'AAAA','gold':'table_name1'},{'sheet_name':'BBBB','gold':'table_name2'},{'sheet_name':'CCCC','gold':'table_name3'},{'sheet_name':'DDDD','gold':'table_name4'}]},{'file_name':'sampledummie_parameters.xlsx','sheets':[{'sheet_name':'AAAA','gold':'table_name1'},{'sheet_name':'BBBB','gold':'table_name2'},{'sheet_name':'CCCC','gold':'table_name2'},{'sheet_name':'DDDD','gold':'table_name3'}]}]}",
"notebookSilver":"path/to/my/notebook"
}
},
"timeout_seconds":36000
}
],
"webhook_notifications":{
"on_start":[
{
"id":"webhook_id_hash"
}
],
"on_failure":[
{
"id":"webhook_id_hash"
}
],
"on_success":[
{
"id":"webhook_id_hash"
}
]
},
"access_control_list":[
{
"group_name":"users",
"permission_level":"CAN_VIEW"
}
]
}

The notebooks and files parameters need to be strings because that is how my notebook process them.

Re: API error when creating serverless job "MALFORMED_REQUEST"

Alberto_Umana — Thu, 23 Jan 2025 14:02:05 GMT

Hi @p_hcarvalho,

Thanks for your question, here's a corrected version of your JSON payload, can you try it?

{
"run_name": "my-other-run-name",
"tasks": [
{
"task_key": "my-other-task-name",
"notebook_task": {
"notebook_path": "my/notebook/path",
"source": "WORKSPACE",
"base_parameters": {
"executionUUID": "d171a035-fe97-46c3-873f-14d91bede7e1",
"pathOutputLanding": "/path/to/my/output/file/sampledummie0/",
"notebooks": "{\"notebooks\": [\"path/to/notebook1\", \"path/to/notebook2\", \"path/to/notebook3\", \"path/to/notebook4\", \"path/to/notebook5\", \"path/to/notebook6\", \"path/to/notebook7\"]}",
"files": "{\"files\": [{\"file_name\": \"sampledummie_base.xlsx\", \"sheets\": [{\"sheet_name\": \"AAAA\", \"gold\": \"table_name1\"}, {\"sheet_name\": \"BBBB\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"CCCC\", \"gold\": \"table_name3\"}, {\"sheet_name\": \"DDDD\", \"gold\": \"table_name4\"}]}, {\"file_name\": \"sampledummie_parameters.xlsx\", \"sheets\": [{\"sheet_name\": \"AAAA\", \"gold\": \"table_name1\"}, {\"sheet_name\": \"BBBB\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"CCCC\", \"gold\": \"table_name2\"}, {\"sheet_name\": \"DDDD\", \"gold\": \"table_name3\"}]}]}",
"notebookSilver": "path/to/my/notebook"
}
},
"timeout_seconds": 36000
}
],
"webhook_notifications": {
"on_start": [
{
"id": "webhook_id_hash"
}
],
"on_failure": [
{
"id": "webhook_id_hash"
}
],
"on_success": [
{
"id": "webhook_id_hash"
}
]
},
"access_control_list": [
{
"group_name": "users",
"permission_level": "CAN_VIEW"
}
]
}

Re: API error when creating serverless job "MALFORMED_REQUEST"

p_hcarvalho — Thu, 23 Jan 2025 15:34:33 GMT

I have tried that solution but it didn't work, any other suggestion?

I'm running it in Azure Data Factory also