Our team are trying to set up a new workspace (8th workspace), but failed to create the storage configurations required for the new workspace with an error of INVALID_STATE: Storage configuration limit exceeded, only 11 storage configurations are allowed.
What we have done:
For every workspace, we need 2 storage configurations (one for the root bucket (https://docs.databricks.com/administration-guide/account-api/aws-storage.html), and one for the audit log bucket (https://docs.databricks.com/administration-guide/account-settings/audit-logs.html)).
On the 18th January, we managed to successfully set up our 8th workspace (along with 16 total storage configurations). However due to some internal issue we decided to take it down and set up our 8th workspace again on the 31st January. This time the above error showed.
We have tried to set up using the Account Console, Databricks Account API, and DatabricksTerraform Provider, and all failed with the same error above.
We suspect that the quota imposed by Databricks has been changed, however from what I found on Databricks' documentation (https://docs.databricks.com/resources/limits.html), the storage configuration limit should be 55 per account. Would someone have more knowledge on this matter please?
Thanks in advance.