- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-14-2023 05:03 PM
Suteja
Thanks a lot for you response
Sorry I was only now that was able to test the solution.
I tried the code you supplied in my Delta Live Table notebook but got an error
Code:
import dlt
from pyspark.sql.functions import *
from pyspark.sql.types import *
from datetime import datetime
import json
custom_params_json = dbutils.widgets.get("custom_params")
custom_params = json.loads(custom_params_json)
runid = custom_params.get("ADFrunid")
Error:
Py4JJavaError: An error occurred while calling o374.getArgument. : com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named custom_params is defined
If I add the widget declaration the code runs without error:
dbutils.widgets.text("custom_params", """{"ADFrunid":"none"}""")
But the notebook does not get the value passed and only resolves the default value
This is the code for the invocation from Azure Data Factory to the API (some key values suppressed for security reasons)
{
"method": "POST",
"headers": {
"Authorization": "Bearer XXXXXXX"
},
"body": "{\n \"fullRefresh\": false,\n \"ADFrunid\": \"6b0a0e74-0f99-463c-9ff0-74fc17ea8958\"\n}"
}
Thanks again for your help
Ruben Sanchez