05-16-2022 05:58 AM
In the current implementation a streaming databricks notebook needs to be started based on the configuration passed. Since the rest of databricks notebooks are being invoked by using ADF,it was decided to use ADF for starting these notebooks. Since there are followup activites that needs to be done after the notebook starts, we tried to start the streaming notebook from an ADF pipeline VIA Rest API.The ADF WEB component is being leveraged to call the rest api for running a notebook. However while trying to invoke the notebook we are getting a Malformed URL Exception :-
{"error_code":"MALFORMED_REQUEST","message":"Invalid JSON given in the body of the request - failed to parse given JSON"}
The below JSON Snippet is being passed as the body for the request
{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX",
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"git_source": null,
"timeout_seconds": 86400
}
Can somebody guide me on what am I doing wrong here
05-17-2022 02:48 AM
@Prasanth KP ,
'{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX"
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"timeout_seconds": 86400
}'
05-17-2022 02:48 AM
@Prasanth KP ,
'{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX"
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"timeout_seconds": 86400
}'
05-18-2022 10:05 PM
05-18-2022 10:07 PM
Hi @Prasanth KP , I'm glad that @Hubert Dudek 's suggestions worked. Would you like to mark his answer as the best?
05-17-2022 02:54 AM
@Prasanth KP ,
clearly, the rest call is invalid. What endpoint do you call?
Also do not forget to authenticate.
May I ask why you use the REST API instead of the available notebook functionality of ADF?
05-18-2022 09:59 PM
Hi @Prasanth KP , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer) and @Werner Stinckens 's responses help you to find the solution? Please let us know.
05-18-2022 10:06 PM
05-18-2022 10:12 PM
Hi @Prasanth KP , Well, In that case would you like to mark @Hubert Dudek 's answer as the best?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.