โ02-14-2022 09:30 AM
If I create a job from the web UI and I select Python wheel, I can add kwargs parameters. Judging from the generated JSON job description, they appear under a section named `namedParameters`.
However, if I use the REST APIs to create a job, it appears I cannot pass namedParameters (link to the documentation).
Am I right that it is not possible to create a Python wheel task with kwargs parameter using REST APIs? Is there any plan to add support?
For context, our use case is that we create jobs using our CI, but we sometimes want to run them manually with different parameters, and we'd like to have the nice kwargs UI to modify the parameters.
โ03-16-2022 01:44 AM
Hi there, the documentation is not up-to-date, you should be able to create such a task with
namedParameters instead of parameters
. Here is a small example:
{
"name": "test_entry_point",
"tasks": [
{
"task_key": "test_entry_point",
"description": "test_entry_point",
"existing_cluster_id": "an-interactive-cluster-that-I-should-probably-not-use",
"python_wheel_task": {
"package_name": "my_package_name",
"entry_point": "my_entry_point",
"named_parameters": {
"param-1": "TODO",
"param-2": "TODO"
}
},
"libraries": [
{
"whl": "dbfs:/TODO"
}
]
}
]
}
I am flagging the fact that the doc is not up-to-date. We will update it soon.
โ02-23-2022 05:38 AM
Hi @Kaniz Fatmaโ, thanks for your answer. I do not understand what is the correlation between the invoke library and Databricks REST APIs. Could you please elaborate?
โ03-16-2022 01:44 AM
Hi there, the documentation is not up-to-date, you should be able to create such a task with
namedParameters instead of parameters
. Here is a small example:
{
"name": "test_entry_point",
"tasks": [
{
"task_key": "test_entry_point",
"description": "test_entry_point",
"existing_cluster_id": "an-interactive-cluster-that-I-should-probably-not-use",
"python_wheel_task": {
"package_name": "my_package_name",
"entry_point": "my_entry_point",
"named_parameters": {
"param-1": "TODO",
"param-2": "TODO"
}
},
"libraries": [
{
"whl": "dbfs:/TODO"
}
]
}
]
}
I am flagging the fact that the doc is not up-to-date. We will update it soon.
โ03-16-2022 04:45 AM
Hi @Rajeev Kumarโ , I've tried and it works, thanks!
โ03-16-2022 04:46 AM
happy to help.
Great Day Ahead!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group