Unity catalog - Service Principal SCIM API account unauthorized

[yvuignie]

Hi,

Is it possible to create groups at the account level in Unity Catalog as a Service Principal ?

I can manage to create groups when authenticated as a user, but not as a Service Principal. I then get an error "user not authorized".

The service principal has the role Account admin visible in the account console and can create other workspace's resources related, as well as metastore using the terraform provider with the host provided as the url of a workspace (but can't manage to use the provider with host https://accounts.azuredatabricks.net, kind of similar issue as https://community.databricks.com/s/question/0D58Y000098lPUkSAM/uc-service-principalterraform).

I tried with terraform as well as Postman via SCIM API 2.0 (Accounts) ({{baseUrl}}/accounts/:account_id/scim/v2/Groups) using the token generated with "az account get-access-token"

The error with terraform:

"Error: cannot create group: User not authorized. Using azure-client-secret auth: host=https://accounts.azuredatabricks.net, account_id=..."

I've read the documentation here: https://docs.microsoft.com/en-us/azure/databricks/administration-guide/users-groups/groups, but haven't found anything related to a service principal restriction.

Thanks for your help

[User16741082858]

Hi @Yannick Vuignier​!

What is the user attribute/role stored in AAD? Make sure that the service principal is assigned the Contributor or Owner role in your Azure portal

[yvuignie]

Hi @Pearl Ubaru​ ,

Thank you for your answer.

The service principal has the role Owner of the subscription.

[User16741082858]

Okay, no problem. So you cannot authenticate into the accounts console yet. We will soon preview Oauth tokens but not sure when. You can add service principals and give them account admin rights by using SCIM tokens. You can also add groups as well via SCIM, as long as you are the account owner or account admin. Here is a document that might be helpful - https://docs.databricks.com/administration-guide/users-groups/service-principals.html#assign-account...

[yvuignie]

Thank you for your help. But the service principal is indeed "Account admin", the tag appears in the account console, the tab is on. Actually we want to use terraform to create groups using this service principal, but as it doesn't work, we tried directly with the API and we get the same result, "User not authorized".

[User16741082858]

Of course! Are you using identity federation?

[yvuignie]

Yes, we have identity federation between the account and the workspaces.

We also have user provisioning enabled. We can manage to create groups with the SCIM token generated from the account console. If user provisioning is enabled, does this means that it is then required to use the SCIM token generated from the account console and that we can't use a service principal to manage groups?

Edit: Actually, it shouldn't be the case since I can create groups with my user using the SCIM API.

[User16741082858]

Hi @Yannick Vuignier​,

You can use the permission assignment APIs - use PATs backed by SPs (https://api-docs.databricks.com/rest/latest/permission-assignment-account-api.html)

[yvuignie]

Hello,

Thank you but I'm sorry I don't see how this API can help adding groups at the account level. Could you maybe please explain a bit ?

[User16741082858]

Hi! the issue is that you cannot create groups at the account level, right? You tried creating with Terraform but had an auth error. You tried with the API and still got an error code.

So the document I shared is for you is to authorize the service principal through the permissions API. I did find out however that it is not possible to do headless auth to the accounts console.

Please let me know if this makes sense!

[yvuignie]

Thank you again for your answer! Yes you understand the issue well, terraform and the api is working with a user but not with a service principal.

But the permission assignment account API is unfortunately workspace related, all endpoints ask for a workspace_id, for instance this description says "Create or update workspace permissions for a principal". What is strange is that the service principal has the role "Account admin".

[Thilo]

We do see the same problem. Any chance that headless authentication into the account will be made possible soon? Otherwise, it does not make sense to have "Account Admin" service principals.

[Anonymous]

Hey @Yannick Vuignier​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[yvuignie]

Hi @Vidula Khanna​ ,

Thank you for your message, but no I am still not able to create groups using a service principal, same as before.

[Anonymous]

Hi @Yannick Vuignier​ 

Sorry for the inconvenience. I will forward your query to the respective person.

Thanks for your patience.