Terminating cluster programmatically

[MichaelO]

Is there any python script that allows me to terminate (not delete)  a cluster in the notebook, similar to this R equivalent of

terminate_cluster(cluster_id, workspace, token = NULL, verbose = T, ...)

[Kaniz · Accepted solution]

Hi @MichaelO, Yes, you can use the Databricks Python API for that. Here is an example Python function that terminates a cluster given a cluster id:

import requests
import json
import time

def terminate_cluster(cluster_id):
    # Define the endpoint for terminating a cluster
    endpoint = f"/api/2.0/clusters/delete"
    
    # Define the URL for the Databricks API
    domain = 'https://your_azure_databricks_instance/api'
    token = 'your_access_token'

    headers = {'Content-Type': 'application/json;charset=UTF-8',
                'Authorization': f'Bearer {token}'}
    
    # Define the JSON payload for terminating the cluster
    payload = {
        "cluster_id": cluster_id,
        "terminate": True
    }

    # Send the request to the Databricks API
    response = requests.post(domain + endpoint, headers=headers, data=json.dumps(payload))

    # Check if the request was successful
    if response.status_code == 200:
        print(f"Cluster {cluster_id} terminated successfully.")
    else:
        print(f"Failed to terminate Cluster {cluster_id}.")
        print(response.content)

You can call this function by passing the cluster_id as a parameter like this:

terminate_cluster("my_cluster_id")

Note: The token the parameter should be replaced by your Databricks personal access token and the domain parameter should be replaced by your domain name.