cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Configure cluster single node with service principal in Azure

wallystart
New Contributor III

Hi! We can configure a cluster single node with single user as service principal using this command:

databricks clusters create --json '
{
    "cluster_name": "my-cluster",
    "spark_version": "13.3.x-scala2.12",
    "node_type_id": "Standard_DS3_v2",
    "driver_node_type_id": "Standard_DS3_v2",
    "single_user_name": "<CLIENTID>",
    "data_security_mode": "SINGLE_USER",
    "runtime_engine": "STANDARD",
    "num_workers": 0,
    "spark_conf": {
     "spark.databricks.cluster.profile":"singleNode",
     "spark.master":"local[*, 4]"
    },
    "custom_tags": {
        "ResourceClass":"SingleNode"
    }
}
'

 But when we want to use this cluster in a notebook we get this error:

Single-user check failed: user '<reserved>@<reserved>.com' attempted to run a command on single-user cluster <CLUSTERID>, but the single user of this cluster is '<CLIENTID>'

We want to get a cluster with a service principal because we know that multiple people may use the cluster with the correct configuration. What we don't know is that it may be missing configuration. In Settings -> Identity and Access -> Service Principal -> Permissions we are managers and users.

If anyone knows we would appreciate it

3 REPLIES 3

Walter_C
Databricks Employee
Databricks Employee

Hello has this user been given with the CAN_BIND permission?

The manager role for SP does not allow users to run notebooks on behalf this clusters as this is part of private preview, to make if work you need to grant can bind permission as follow:

$ vim grant-service-principal.json
{
  "access_control_list": [
    {
      "user_name": "{username}",
      "permission_level": "CAN_BIND"
    }
  ]
}

$ curl -X PATCH  {DATABRICKS_HOST}/api/2.0/permissions/service-principals/{SP_ID} --header "Content-type: application/json" --header "Authorization: Bearer ${DATABRICKS_TOKEN}" --data @Grant-service-principal.json

wallystart
New Contributor III

Hello Walter_C! Thanks for your answer

We ran this command but got the error "ENDPOINT_NOT_FOUND". Do we need to activate any preview function? Do we have any mistake?

wallystart_0-1734384923666.png

Thanks

Walter_C
Databricks Employee
Databricks Employee

Can you share an screenshot of the cluster on the UI side please, if you are creating a cluster assigned to a service principal this means that it should have already the Private preview enabled.

If the private preview is not enabled it will throw the error message you got.