Walter_C
Databricks Employee
Databricks Employee

To create a serverless job using the API, you no longer need to specify one of new_cluster, existing_cluster_id, or job_cluster_key in each task. Instead, only tasks with task_key and your task to run is required. Here is an example of how you can create a serverless job:

{
 "name": "Serverless Job",
 "tasks": [
  {
   "task_key": "My_task",
   "python_wheel_task": {
    "package_name": "databricks_jaws",
    "entry_point": "run_analysis",
    "named_parameters": {
     "dry_run": "true"
    }
   },
   "environment_key": "my-serverless-compute"
  }
 ],
 "tags": {
  "department": "sales"
 },
 "environments": [
  {
   "environment_key": "default",
   "spec": {
    "client": "1",
    "dependencies": [
     "/Volumes/<catalog>/<schema>/<volume>/<path>.whl",
     "/Workspace/my_project/dist.whl",
     "simplejson",
     "-r /Workspace/my_project/requirements.txt"
    ]
   }
  }
 ]
}

In this example, the job is named "Serverless Job" and it has a single task with the key "My_task". The task is a Python wheel task that runs the "run_analysis" function from the "databricks_jaws" package. The task is run in the "my-serverless-compute" environment. The job also has a tag indicating that it belongs to the "sales" department.