05-16-2022 05:58 AM
In the current implementation a streaming databricks notebook needs to be started based on the configuration passed. Since the rest of databricks notebooks are being invoked by using ADF,it was decided to use ADF for starting these notebooks. Since there are followup activites that needs to be done after the notebook starts, we tried to start the streaming notebook from an ADF pipeline VIA Rest API.The ADF WEB component is being leveraged to call the rest api for running a notebook. However while trying to invoke the notebook we are getting a Malformed URL Exception :-
{"error_code":"MALFORMED_REQUEST","message":"Invalid JSON given in the body of the request - failed to parse given JSON"}
The below JSON Snippet is being passed as the body for the request
{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX",
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"git_source": null,
"timeout_seconds": 86400
}
Can somebody guide me on what am I doing wrong here
05-17-2022 02:48 AM
@Prasanth KP ,
'{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX"
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"timeout_seconds": 86400
}'
05-17-2022 02:48 AM
@Prasanth KP ,
'{
"tasks": [
{
"task_key": "Match",
"description": "Matches orders with user sessions",
"notebook_task": {
"notebook_path": "/Users/userxxx@xxxxsandbox.com/Demo/RealTimeXXXXXXX"
},
"timeout_seconds": 86400
}
],
"run_name": "A multitask job run",
"timeout_seconds": 86400
}'
05-18-2022 10:05 PM
05-18-2022 10:07 PM
Hi @Prasanth KP , I'm glad that @Hubert Dudek 's suggestions worked. Would you like to mark his answer as the best?
05-17-2022 02:54 AM
@Prasanth KP ,
clearly, the rest call is invalid. What endpoint do you call?
Also do not forget to authenticate.
May I ask why you use the REST API instead of the available notebook functionality of ADF?
05-18-2022 09:59 PM
Hi @Prasanth KP , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer) and @Werner Stinckens 's responses help you to find the solution? Please let us know.
05-18-2022 10:06 PM
05-18-2022 10:12 PM
Hi @Prasanth KP , Well, In that case would you like to mark @Hubert Dudek 's answer as the best?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group