cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Support for kwargs parameter in `/2.1/jobs/create` endpoint for `python_wheel_task`

GabrieleMuciacc
New Contributor III

If I create a job from the web UI and I select Python wheel, I can add kwargs parameters. Judging from the generated JSON job description, they appear under a section named `namedParameters`.

However, if I use the REST APIs to create a job, it appears I cannot pass namedParameters (link to the documentation).

Am I right that it is not possible to create a Python wheel task with kwargs parameter using REST APIs? Is there any plan to add support?

For context, our use case is that we create jobs using our CI, but we sometimes want to run them manually with different parameters, and we'd like to have the nice kwargs UI to modify the parameters.

1 ACCEPTED SOLUTION

Accepted Solutions

rajeev_thakur_c
New Contributor III
New Contributor III

Hi there, the documentation is not up-to-date, you should be able to create such a task with 

namedParameters instead of parameters

. Here is a small example:

{

 "name": "test_entry_point",

 "tasks": [

  {

   "task_key": "test_entry_point",

   "description": "test_entry_point",

   "existing_cluster_id": "an-interactive-cluster-that-I-should-probably-not-use",

   "python_wheel_task": {

    "package_name": "my_package_name",

    "entry_point": "my_entry_point",

    "named_parameters": {

     "param-1": "TODO",

     "param-2": "TODO"

    }

   },

   "libraries": [

    {

     "whl": "dbfs:/TODO"

    }

   ]

  }

 ]

}

I am flagging the fact that the doc is not up-to-date. We will update it soon.

View solution in original post

7 REPLIES 7

Kaniz_Fatma
Community Manager
Community Manager

Hi @Gabriele Muciacciaโ€‹ ! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.

Kaniz_Fatma
Community Manager
Community Manager

Hi @Gabriele Muciacciaโ€‹ , You'll certainly get some insights here:-

GabrieleMuciacc
New Contributor III

Hi @Kaniz Fatmaโ€‹, thanks for your answer. I do not understand what is the correlation between the invoke library and Databricks REST APIs. Could you please elaborate?

rajeev_thakur_c
New Contributor III
New Contributor III

Hi there, the documentation is not up-to-date, you should be able to create such a task with 

namedParameters instead of parameters

. Here is a small example:

{

 "name": "test_entry_point",

 "tasks": [

  {

   "task_key": "test_entry_point",

   "description": "test_entry_point",

   "existing_cluster_id": "an-interactive-cluster-that-I-should-probably-not-use",

   "python_wheel_task": {

    "package_name": "my_package_name",

    "entry_point": "my_entry_point",

    "named_parameters": {

     "param-1": "TODO",

     "param-2": "TODO"

    }

   },

   "libraries": [

    {

     "whl": "dbfs:/TODO"

    }

   ]

  }

 ]

}

I am flagging the fact that the doc is not up-to-date. We will update it soon.

Hi @Rajeev Kumarโ€‹ , I've tried and it works, thanks!

happy to help.

Great Day Ahead!

Kaniz_Fatma
Community Manager
Community Manager

Hi @Gabriele Muciacciaโ€‹ , Does @Rajeev Kumarโ€‹ 's solution answer your question?

If yes, would you like to mark his answer as the best?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group