Create a new cluster in Databricks using databricks-cli

User16765131552 — Fri, 18 Jun 2021 19:22:24 GMT

I'm trying to create a new cluster in Databricks on Azure using databricks-cli.

I'm using the following command:

databricks clusters create --json '{ "cluster_name": "template2", "spark_version": "4.1.x-scala2.11" }'

And getting back this error:

Error: {"error_code":"INVALID_PARAMETER_VALUE","message":"Missing required field: size"}

I can't find documentation on this issue, would be happy to receive some help.

Re: Create a new cluster in Databricks using databricks-cli

User16765131552 — Fri, 18 Jun 2021 19:24:11 GMT

I found the right answer here.

The correct format to run this command on azure is:

databricks clusters create --json '{ "cluster_name": "my-cluster", "spark_version": "4.1.x-scala2.11", "node_type_id": "Standard_DS3_v2", "autoscale" : { "min_workers": 2, "max_workers": 50 } }'

topic Create a new cluster in Databricks using databricks-cli in Data Engineering

Create a new cluster in Databricks using databricks-cli

Re: Create a new cluster in Databricks using databricks-cli