Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-21-2023 11:31 AM
We can update cluster configuration prgramatically using databricks api.
Steps:
1) Fetch all the clusters info present in your workspace.
2) Loop through each cluster info
3) Use the parsed info like cluster id etc.. to update cluster config
Example Psuedo code:
import requests
import json
host = "https://<YOUR-WORKSPACE-URL>"
workspace_access_token = "<YOUR-ACCESS-TOKEN>"
cluster_id = "<YOUR-CLUSTER-ID>"
workers_min = 1
workers_max = 2
response = requests.post(
f"{host}/api/2.0/clusters/edit",
headers={
"Authorization": f"Bearer {workspace_access_token}",
"Content-Type": "application/json"
},
data={
"cluster_id": cluster_id,
"min_workers": workers_min,
"max_workers": workers_max
}
)