cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How exactly to create cluster policy via Databricks CLI ?

guangyi
Contributor III

I tried these ways they are all not working:

 

 

Save the json config into a JSON file locally and run 

databricks cluster-policies create --json cluster-policy.json 

Error message: Error: invalid character 'c' looking for beginning of value



Save the json config into a JSON file locally and run

databricks cluster-policies create --json @"cluster-policy.json"

Error message: Error: 'definition' must be supplied.



Try using `definition` and run:

databricks cluster-policies create --definition {"cluster_type":{"type":"fixed","value":"job"}}

Error message: Error: Definition has invalid JSON



Give a more detailed `definition` and run 

databricks cluster-policies create --definition {"spark_conf.spark.databricks.cluster.profile":{"type":"forbidden","hidden":true},"spark_version":{"type":"unlimited","defaultValue":"auto:latest-lts"},"node_type_id":{"type":"unlimited","defaultValue":"Standard_DS3_v2","isOptional":true},"num_workers":{"type":"unlimited","defaultValue":4,"isOptional":true},"azure_attributes.availability":{"type":"unlimited","defaultValue":"SPOT_WITH_FALLBACK_AZURE"},"azure_attributes.spot_bid_max_price":{"type":"fixed","value":100,"hidden":true},"instance_pool_id":{"type":"forbidden","hidden":true},"driver_instance_pool_id":{"type":"forbidden","hidden":true},"cluster_type":{"type":"fixed","value":"job"}}

Error message: Error: accepts 1 arg(s), received 20

1 ACCEPTED SOLUTION

Accepted Solutions

szymon_dybczak
Esteemed Contributor III

Hi @guangyi ,

Try to add @ before the name of json file

databricks cluster-policies create --json @policy.json

 

Also make sure that you're escaping quotation marks like they do in below documenation:

Create a new policy | Cluster Policies API | REST API reference | Azure Databricks

 

{
  "name": "Test policy",
  "definition": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n"
}

 


In my case after these changes I was able to create policy:

Slash_0-1722323175186.png

 



View solution in original post

1 REPLY 1

szymon_dybczak
Esteemed Contributor III

Hi @guangyi ,

Try to add @ before the name of json file

databricks cluster-policies create --json @policy.json

 

Also make sure that you're escaping quotation marks like they do in below documenation:

Create a new policy | Cluster Policies API | REST API reference | Azure Databricks

 

{
  "name": "Test policy",
  "definition": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n"
}

 


In my case after these changes I was able to create policy:

Slash_0-1722323175186.png

 



Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now