Can anyone tell me how to submit a job using serverless compute using restapi

[tonyd]

#serverless#createjob

[filipniziol]

Hi @tonyd ,
If you want to use serverless, first of all you need to enable it:

When enabled, when sending the request to the REST API simply do not specify the cluster:

import requests
import json

# Your Databricks domain
DATABRICKS_DOMAIN = '<>'
# Personal Access Token for authentication
TOKEN = '<>'

# The JSON string representing your job configuration
job_json_string =  """{
  "name": "my_serverless_job",
  "email_notifications": {
    "no_alert_for_skipped_runs": false
  },
  "webhook_notifications": {},
  "timeout_seconds": 0,
  "max_concurrent_runs": 1,
  "tasks": [
    {
      "task_key": "df_regular",
      "run_if": "ALL_SUCCESS",
      "notebook_task": {
        "notebook_path": "/Workspace/Users/my_user/df_regular",
        "base_parameters": {
          "storage_account": "my_storage_account"
        },
        "source": "WORKSPACE"
      },
      "timeout_seconds": 0,
      "email_notifications": {},
      "notification_settings": {
        "no_alert_for_skipped_runs": false,
        "no_alert_for_canceled_runs": false,
        "alert_on_last_attempt": false
      },
      "webhook_notifications": {}
    }
  ],
  "run_as": {
    "user_name": "my_user"
  }
}"""

# Convert the JSON string to a dictionary
job_config = json.loads(job_json_string)

# The API endpoint for creating a job
api_url = f'{DATABRICKS_DOMAIN}/api/2.1/jobs/create'

# Headers for the request
headers = {
    'Authorization': f'Bearer {TOKEN}',
    'Content-Type': 'application/json'
}

# Make the POST request to create the job
response = requests.post(api_url, headers=headers, json=job_config)

# Check if the request was successful
if response.status_code == 200:
    # Job created successfully
    print("Job created successfully.")
    # Print the job ID
    job_id = response.json()['job_id']
    print(f"Job ID: {job_id}")
else:
    # There was an error
    print("Failed to create job.")
    print(f"Status Code: {response.status_code}")
    print(f"Response: {response.text}")

The job has been created:

The job compute is serverless:

Hope it helps