Unable to access Live Ganglia UI

[arkadiuszr]

I’m having a trouble accessing Ganglia UI on a job run cluster. The job run is executed by a service principal and I would like to enable the Ganglia UI view to an user/admin group in Databricks.

The error I get is: HTTP ERROR 403 PERMISSION_DENIED: Cannot use the cluster as user (my_email@email.com) must have at least 'Bind' permission on the service principal.

The situation occurs only during job run, screenshots taken from Ganglia UI are accessible.Following the documentation, I’ve tried to set access_control_list property. When I set access_control_list to [{"user_name": "my_email@email.com", "permission_level": "CAN_VIEW"}] then request was accepted (200), but Ganglia UI is still not accessible, however when I’ve tried [{"user_name": "my_email@email.com", "permission_level": "CAN_BIND"}] then I got Bad Request (400) and a message Permission type not defined. https://docs.databricks.com/api/workspace/jobs/submit

Also CAN_BIND value is not described in this documentation: https://docs.databricks.com/en/security/auth-authz/access-control/jobs-acl.html so I'm a little confused.

I’m also not sure I have chosen the right path since Ganglia UI can also belong to the cluster rather than to a job. Can you point me in a right direction?

[shan_chandra · Accepted solution]

Hi @arkadiuszr  -  can you please try the following steps and let us know?

To resolve this error, first you want to add your service principal to the workspace:

1. Go to admin console as a workspace admin
2. Select “service principal tab”
3. Click “Add service principal button”
4. Select the account service principals in the dropdown and add them into this workspace.

then bind your user to the service principal:

• You must have the `can_bind` permission on a service principal.
• If you are a workspace admin you will already have this permission.

To grant this permission to users, the workflow is as follows:

1. As a workspace admin, goto the Admin Console.
2. Get or create a PAT token.
3. Get or create a service principal in the workspace.
4. Get the id of the service principal, this should look like a GUID and not the descriptive name.
5. Grant permission for your user to bind to that service principal.

$ vim grant-service-principal.json
{
"access_control_list": [
{
"user_name": "{username}",
"permission_level": "CAN_BIND"
}
]
}
$ curl -X PATCH {DATABRICKS_HOST}/api/2.0/permissions/service-principals/{SP_ID}
header "Content-type: application/json" header "Authorization: Bearer
${DATABRICKS_TOKEN}" data @grant-service-principal.json

Reference on Service Principal: https://docs.databricks.com/administration-guide/users-groups/service-principals.html#what-is-a-serv...

[arkadiuszr]

Thank you for the time you spent to clarify a few things to me. Yes it worked indeed with one remark {SP_ID} is not the GUID but a numeric one instead. To fetch it I. had to make a call to that endpoint "api/2.0/preview/scim/v2/ServicePrincipals"

Do you know if there is a terraform component that allows that? I've searched a little and couldn't find.

[shan_chandra]

@arkadiuszr - Can you please check the documentation for SP here for examples - https://docs.databricks.com/en/dev-tools/service-principals-tools-apis.html ?

[arkadiuszr]

Ok, there is no terraform component that does that yet. https://docs.databricks.com/en/dev-tools/service-principals-tools-apis.html

https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/service_principa...

Thanks

[reddybhargava]

@shan_chandra 

Thank you! We encountered the same issue, and your solution was very helpful in resolving it. We have a follow-up question: how can we revoke the permissions granted through this method using the REST API? I couldn’t find any REST API methods for revoking or deleting object permissions.