cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Provisioning Unity Metastore for each environment using Terraform

mathiaskvist
New Contributor III

Hi

Unity Metastore can currently only be provisioned once for each Azure region. I have three environments in Azure each consisting of a separate Subscription and Resource Group (RG). Each RG contains a Databricks Workspace and a Storage Account holding data. This setup is provisioned using Terraform and each environment has its' own state file. Thus I would like to provision a Unity Metastore for each environment, which is not possible, since there can only be one metastore per region.

This means that only one of my environments can provision the Unity Metastore which will lead to my Terraform code being a huge mess as it should be environment aware. E.g. when running Terraform for the dev environment it would create the metastore and when running for production it would import the metastore as a Terraform data source (or vice versa). 

What do others do to handle this issue?

2 REPLIES 2

Corbin
Databricks Employee
Databricks Employee

You're not wrong. How you've described it is currently the only way to do it with TF. This should be resolved once the account-level API for UC is incorporated into TF (see PR here). Then you'd have 4 statefiles, 1 for account-level tf state and 3 for dev/test/prod. Until that day, you're correct that you need to create in one and reference in the others. Quite clunky and definitely looking forward to account-level UC API integrated into TF.

Corbin
Databricks Employee
Databricks Employee

Ask and ye shall receive: https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.24.0

They've implemented account-level UC API in terraform now!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group