Terraform databricks_storage_credential has wrong External ID

dvmentalmadess
Valued Contributor

We create storage credentials using Terraform. I don't see any way to specify a given External ID (DBR Account ID) when creating the credentials via Terraform or in the web UI console. However, today when I tried creating a new set of credentials using the same custom Terraform module we've used for every other storage credential, the external ID assigned to the credentials is different from all of our other storage credentials.

As a result, I'm unable to use the storage credentials. I get forbidden errors from S3 whenever I try to create a new external location using our terraform module. I am completely blocked from adding new sources to our production unity catalog using our automation. NOTE: this does not happen if I manually create credentials using the web UI.

Screenshot 2023-11-28 at 6.37.50 PM.png

 While I could incorporate the "new" external id into my IAM policies, there is no way to get it from the Terraform resource attributes after it has been created. Also, the IAM role must be created before the storage credential. This seems like a bug, but I can't find any other instance of this happening elsewhere.

UPDATE: It appears the External ID of the storage credential is now different each time I re-create it (destroy then create).