How exactly to create cluster policy via Databricks CLI ?

guangyi — Tue, 30 Jul 2024 05:45:37 GMT

I tried these ways they are all not working:

Save the json config into a JSON file locally and run

databricks cluster-policies create --json cluster-policy.json

Error message: Error: invalid character 'c' looking for beginning of value

Save the json config into a JSON file locally and run

databricks cluster-policies create --json @"cluster-policy.json"

Error message: Error: 'definition' must be supplied.

Try using `definition` and run:

databricks cluster-policies create --definition {"cluster_type":{"type":"fixed","value":"job"}}

Error message: Error: Definition has invalid JSON

Give a more detailed `definition` and run

databricks cluster-policies create --definition {"spark_conf.spark.databricks.cluster.profile":{"type":"forbidden","hidden":true},"spark_version":{"type":"unlimited","defaultValue":"auto:latest-lts"},"node_type_id":{"type":"unlimited","defaultValue":"Standard_DS3_v2","isOptional":true},"num_workers":{"type":"unlimited","defaultValue":4,"isOptional":true},"azure_attributes.availability":{"type":"unlimited","defaultValue":"SPOT_WITH_FALLBACK_AZURE"},"azure_attributes.spot_bid_max_price":{"type":"fixed","value":100,"hidden":true},"instance_pool_id":{"type":"forbidden","hidden":true},"driver_instance_pool_id":{"type":"forbidden","hidden":true},"cluster_type":{"type":"fixed","value":"job"}}

Error message: Error: accepts 1 arg(s), received 20

Re: How exactly to create cluster policy via Databricks CLI ?

szymon_dybczak — Tue, 30 Jul 2024 07:06:59 GMT

Hi @guangyi ,

Try to add @ before the name of json file

databricks cluster-policies create --json @policy.json

Also make sure that you're escaping quotation marks like they do in below documenation:

Create a new policy | Cluster Policies API | REST API reference | Azure Databricks

{ "name": "Test policy", "definition": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n" }

In my case after these changes I was able to create policy:

topic Re: How exactly to create cluster policy via Databricks CLI ? in Data Engineering

How exactly to create cluster policy via Databricks CLI ?

Re: How exactly to create cluster policy via Databricks CLI ?