cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Create Storage Credential 500 Response

ledbutter
New Contributor II

I'm trying to create storage credentials for an Azure Databricks Connector at the workspace level with a service principal that has the CREATE_STORAGE_CREDENTIAL but is NOT an account admin. For this test, the SP has the owner role on the connector.

I've tried this via Terraform, Databricks CLI and Databricks CLI API with a 500 response (or an error with no message) in all cases.

Simplified Terraform:

 

resource "azurerm_databricks_access_connector" "managed" {
  name = format("ac-%s-%s", local.oldw_environment, local.business_unit)
  resource_group_name = data.azurerm_resource_group.oldw.name
  location = data.azurerm_resource_group.oldw.location

  identity {
    type = "SystemAssigned"
  }
}

 

CLI call:

 

databricks storage-credentials create --json '{"name": "manual_test", "azure_managed_identity": {"access_connector_id": "/subscriptions/$MY_SUB_ID/resourceGroups/$MY_RG/providers/Microsoft.Databricks/accessConnectors/manual-test"}, "skip_validation": true}' -p $MY_PROFILE --debug

 

CLI response:

 

15:11:11  INFO start pid=38926 version=0.211.0 args="databricks, storage-credentials, create, --json, {\"name\": \"manual_test\", \"azure_managed_identity\": {\"access_connector_id\": \"/subscriptions/$MY_SUB_ID/resourceGroups/$MY_RG/providers/Microsoft.Databricks/accessConnectors/manual-test\"}, \"skip_validation\": true}, -p, dev, --debug"
15:11:11 DEBUG Loading dev profile from /Users/sgrosen/.databrickscfg pid=38926 sdk=true
15:11:11  INFO Generating AAD token for Service Principal ($SP_ID) pid=38926 sdk=true
15:11:11 DEBUG POST /$TENANT_ID/oauth2/token
> [non-JSON document of 19 bytes]. <http.RoundTripper>
< HTTP/1.1 200 OK
< {
<   "access_token": "**REDACTED**",
<   "expires_in": "3599",
<   "expires_on": "1707775871",
<   "ext_expires_in": "3599",
<   "not_before": "1707771971",
<   "resource": "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d",
<   "token_type": "Bearer"
< } pid=38926 sdk=true
15:11:12 DEBUG POST /$TENANT_ID/oauth2/token
> [non-JSON document of 19 bytes]. <http.RoundTripper>
< HTTP/1.1 200 OK
< {
<   "access_token": "**REDACTED**",
<   "expires_in": "3599",
<   "expires_on": "1707775871",
<   "ext_expires_in": "3599",
<   "not_before": "1707771971",
<   "resource": "https://management.core.windows.net/",
<   "token_type": "Bearer"
< } pid=38926 sdk=true
15:11:12 DEBUG non-retriable error:  pid=38926 sdk=true
15:11:12 DEBUG POST /api/2.1/unity-catalog/storage-credentials
> {
>   "azure_managed_identity": {
>     "access_connector_id": "/subscriptions/$MY_SUB_ID/resourceGroups/$MY_RG/pro... (56 more bytes)"
>   },
>   "name": "manual_test",
>   "skip_validation": true
> }
< HTTP/2.0 500 Internal Server Error
< {
<   "details": [
<     {
<       "@type": "type.googleapis.com/google.rpc.RequestInfo",
<       "request_id": "3e023b35-1fdf-4004-bd87-b5b3ef54e4df",
<       "serving_data": ""
<     }
<   ],
<   "error_code": "INTERNAL_ERROR",
<   "message": ""
< } pid=38926 sdk=true
Error:
15:11:12 ERROR failed execution pid=38926 exit_code=1 error=

 

 

Any ideas on what is going wrong here? Up until this point, the only storage credentials created have been created manually by account admins. Our group is trying to automate this process but running into this problem.

1 ACCEPTED SOLUTION

Accepted Solutions

Ayushi_Suthar
Databricks Employee
Databricks Employee

Hi @ledbutter , hope you are doing well today! 

I have gone through the details and this issue might be related to https://github.com/databricks/cli/issues/1080

Please refer to this for more details: https://github.com/databricks/cli/issues/1108

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi

View solution in original post

2 REPLIES 2

Ayushi_Suthar
Databricks Employee
Databricks Employee

Hi @ledbutter , hope you are doing well today! 

I have gone through the details and this issue might be related to https://github.com/databricks/cli/issues/1080

Please refer to this for more details: https://github.com/databricks/cli/issues/1108

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi

Thanks for the links! I will watch those issues.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group