cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Provisioning Unity Metastore for each environment using Terraform

mathiaskvist
New Contributor III

Hi

Unity Metastore can currently only be provisioned once for each Azure region. I have three environments in Azure each consisting of a separate Subscription and Resource Group (RG). Each RG contains a Databricks Workspace and a Storage Account holding data. This setup is provisioned using Terraform and each environment has its' own state file. Thus I would like to provision a Unity Metastore for each environment, which is not possible, since there can only be one metastore per region.

This means that only one of my environments can provision the Unity Metastore which will lead to my Terraform code being a huge mess as it should be environment aware. E.g. when running Terraform for the dev environment it would create the metastore and when running for production it would import the metastore as a Terraform data source (or vice versa). 

What do others do to handle this issue?

2 REPLIES 2

Corbin
New Contributor III
New Contributor III

You're not wrong. How you've described it is currently the only way to do it with TF. This should be resolved once the account-level API for UC is incorporated into TF (see PR here). Then you'd have 4 statefiles, 1 for account-level tf state and 3 for dev/test/prod. Until that day, you're correct that you need to create in one and reference in the others. Quite clunky and definitely looking forward to account-level UC API integrated into TF.

Corbin
New Contributor III
New Contributor III

Ask and ye shall receive: https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.24.0

They've implemented account-level UC API in terraform now!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!