Terraform: Grant account-level group access to instance profile

[dvmentalmadess]

We have setup SCIM with Okta at the account-level and setup Unity Catalog and are in the process of migrating groups from workspace-local to account-level. I have an instance profile that was assigned to a workspace-local group. using `databricks_group_instance_profile`. I'm trying to grant the account-level group access to the instance profile (the profile grants access to resources other than S3) and both `databricks_group_instance_profile` and `databricks_group_role` return the following error:

cannot create group instance profile: invalidSyntax Groups attribute cannot be updated as group [name] can only be managed in account.

Or

cannot create group role: invalidSyntax Groups attribute cannot be updated as group [name] can only be managed in account.

If I set the provider to point to accounts.cloud.databricks.com then I get:

cannot create group role: invalidValue Invalid role value [arn]

Or

cannot create group instance profile: invalidValue Invalid role value [arn]

Finally, on a lark I tried to create the `databricks_instance_profile` using the accounts.cloud.databricks.com host and that didn't work either (as expected):

cannot create instance profile: HTTP method POST is not supported by this URL

Using `databricks_group_instance_profile` using the workspace host will actually set the permission successfully even though it throws an error. However, because of the error the resource is never added to Terraform state and `terraform apply` fails anyway so it wouldn't matter anyhow.

Granting permission to the group on the instance profile works fine manually if I use the workspace UI, but that is only a temporary fix since we manage our entire workspace in terraform. How am I supposed to get this to work?

Terraform: v1.4.2

Databricks provider(s): v1.13.0, v1.14.3

[Anonymous]

@Mark Miller​ :

The error you are facing suggests that the group you are trying to manage can only be managed at the account level, which means you cannot manage it at the workspace level. This error typically occurs when you attempt to assign a group to an instance profile or role that is outside the scope of the workspace.

To resolve this issue, you can try the following:

1. Ensure that the group you are trying to manage is at the account level, and not at the workspace level.
2. Update the provider version to the latest version to ensure you have access to any new fixes or updates that may address this issue.
3. Try using the databricks_group resource instead of the databricks_group_instance_profile or databricks_group_role resources. This resource allows you to create and manage groups at the account level, and it can be used to grant access to instance profiles and roles.

Here's an example of how you could create a group and grant it access to an instance profile:

resource "databricks_instance_profile" "example" {
  name = "example-instance-profile"
}
 
resource "databricks_group" "example" {
  display_name = "example-group"
  provider = databricks.accounts
}
 
resource "databricks_group_instance_profile" "example" {
  group_id = databricks_group.example.id
  instance_profile_arn = databricks_instance_profile.example.arn
}

This creates a new instance profile and group at the account level, and then grants the group access to the instance profile.

[Anonymous]

Hi @Mark Miller​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[dvmentalmadess]

1. All of our groups are account-level groups (see opening paragraph)
2. Provider version was latest at the time of writing
3. I don't understand what this is trying to say. You can't use databricks_group in place of databricks_group_instance_profile or databricks_group_role. They can only be used together as shown in your example. This is exactly the first thing I tried and I get the first error I listed in my post.

[dvmentalmadess · Accepted solution]

Retried this using `databricks_group_role` after the `1.210` release of the `databricks/databricks` provider. This worked with an account-level group using the workspace provider and credentials.