Hi ,
We have an issue with our intial setup and design.
We are using a single Azure, Premium, Block blob storage account with hierarchical namespace and LRS enabled.
We have three containers within it, one for each environment – Dev -Test – Prod
But the issue we encounter is that when we try to add our secondary Databricks workspace (Test) and connect into the Unity catalog we cannot target the Test container within the storage account.
So this new test workspace can only view the Dev container (The DEV Workspace was setup first) and not its own Test container.
Is there a way to have all three of our databricks workspaces looking at a single azure storage account and targeting their respective containers within the storage account?
Test WS > Test container
DEV WS > Dev container
Prd WS > container
As you know we cannot use more than one metastore in the same region so we cant use a separate storage account for each environment as a workaround.
any help here would be great.
thanks
