Configure cluster single node with service principal in Azure

[wallystart]

Hi! We can configure a cluster single node with single user as service principal using this command:

databricks clusters create --json '
{
    "cluster_name": "my-cluster",
    "spark_version": "13.3.x-scala2.12",
    "node_type_id": "Standard_DS3_v2",
    "driver_node_type_id": "Standard_DS3_v2",
    "single_user_name": "<CLIENTID>",
    "data_security_mode": "SINGLE_USER",
    "runtime_engine": "STANDARD",
    "num_workers": 0,
    "spark_conf": {
     "spark.databricks.cluster.profile":"singleNode",
     "spark.master":"local[*, 4]"
    },
    "custom_tags": {
        "ResourceClass":"SingleNode"
    }
}
'

 But when we want to use this cluster in a notebook we get this error:

Single-user check failed: user '<reserved>@<reserved>.com' attempted to run a command on single-user cluster <CLUSTERID>, but the single user of this cluster is '<CLIENTID>'

We want to get a cluster with a service principal because we know that multiple people may use the cluster with the correct configuration. What we don't know is that it may be missing configuration. In Settings -> Identity and Access -> Service Principal -> Permissions we are managers and users.

If anyone knows we would appreciate it

[Walter_C]

Hello has this user been given with the CAN_BIND permission?

The manager role for SP does not allow users to run notebooks on behalf this clusters as this is part of private preview, to make if work you need to grant can bind permission as follow:

$ vim grant-service-principal.json
{
  "access_control_list": [
    {
      "user_name": "{username}",
      "permission_level": "CAN_BIND"
    }
  ]
}

$ curl -X PATCH  {DATABRICKS_HOST}/api/2.0/permissions/service-principals/{SP_ID} --header "Content-type: application/json" --header "Authorization: Bearer ${DATABRICKS_TOKEN}" --data @Grant-service-principal.json

[wallystart]

Hello Walter_C! Thanks for your answer

We ran this command but got the error "ENDPOINT_NOT_FOUND". Do we need to activate any preview function? Do we have any mistake?

Thanks

[Walter_C]

Can you share an screenshot of the cluster on the UI side please, if you are creating a cluster assigned to a service principal this means that it should have already the Private preview enabled.

If the private preview is not enabled it will throw the error message you got.