Adding a service principal to workspace fails

[bas]

Trying to add an Azure Service Principal to a workspace fails, whether via the web UI or the databricks cli. With the databricks, the console gives this:

databricks service-principals -p ************* create --application-id ********-****-****-****-************ --display-name ****************** --debug

18:28:51  INFO start pid=17186 version=0.212.2 args="databricks, service-principals, -p, *************, create, --application-id, ********-****-****-****-************, --display-name, ******************, --debug"
18:28:52 DEBUG non-retriable error: INTERNAL_ERROR: 4 (of class java.lang.Integer) pid=17186 sdk=true
18:28:52 DEBUG POST /api/2.0/preview/scim/v2/ServicePrincipals
> {
>   "applicationId": "********-****-****-****-************",
>   "displayName": "******************"
> }
< HTTP/2.0 500 Internal Server Error
< {
<   "detail": "INTERNAL_ERROR: 4 (of class java.lang.Integer)",
<   "schemas": [
<     "urn:ietf:params:scim:api:messages:2.0:Error"
<   ],
<   "status": "500"
< } pid=17186 sdk=true
Error: INTERNAL_ERROR: 4 (of class java.lang.Integer)
18:28:52 ERROR failed execution pid=17186 exit_code=1 error="INTERNAL_ERROR: 4 (of class java.lang.Integer)"

#azure #Identity and access

[Walter_C]

The issue you're experiencing seems to be related to the Databricks provider's authentication configuration in Terraform. Here are some steps that you can follow to potentially resolve the issue:
1. You need to ensure that the Databricks provider is correctly configured with the necessary parameters, especially the host parameter which points to the Databricks workspace URL. Here's how you can configure it:

terraform
provider "databricks" {
  host = "https://${azurerm_databricks_workspace.databricks.workspace_url}"
}

By setting the host parameter to the URL of the Databricks workspace you've created, the provider knows where to direct API calls. Ensure that the azurerm_databricks_workspace.databricks.workspace_url correctly points to the URL of your Databricks workspace.
2. Additionally, make sure that the environment variables ARM_CLIENT_SECRET, ARM_CLIENT_ID, and ARM_TENANT_ID are correctly set in your Git workflow environment. These should correspond to your Service Principal's client secret, client ID, and tenant ID, respectively.

[bas]

I'm not sure I understand the "relationship with terraform" as I tried creating the Service Principal using the Web UI and the Databricks cli. Furthermore, creating Service Principals was working fine two days ago for the very same workspace.

Thanks 

[Walter_C]

Apologize for the confusion so you are also having issues when doing it through UI?

[bas]

Yes, I'm having the same issue through the UI. With exact error message

INTERNAL_ERROR: 4 (of class java.lang.Integer)

[Gerben]

We are experiencing the same here.

[Rania07]

Same here with Azure Databricks

[Gerben · Accepted solution]

I got a response from Azure Support:

"Kindly note that this issue has already been addressed by Databricks Engineering team, and the fix will be deployed by End of the day Pacific time today."

[bas]

I confirm that the issue has been resolved for me. Thanks.

[brickmaker]

Same issue in AWS Databricks.