cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Unity catalog metastore and databricks workspace provisioning

ducng
New Contributor II

 

Hi,
I am using terraform to create a databricks metastore using this.
Now the issue is that I want to use this metastore reference in another terraform project where...

data "databricks_metastore" "metastore" {
  name                = "metastore-name"
} 

 

2 REPLIES 2

Kaniz
Community Manager
Community Manager

Hi @ducng, When working with Databricks metastores in Terraform, you can create a metastore using the databricks_metastore resource. This metastore acts as a top-level container for objects in Unity Catalog, including data assets like tables and views, along with the associated permissions governing access to them. Databricks account admins can create metastores and assign them to Databricks workspaces to control ...12.

Here’s an example of how you can define a databricks_metastore resource in your Terraform configuration:

data "databricks_metastore" "metastore" {
  name          = "metastore-name"
  storage_root  = "s3://${aws_s3_bucket.metastore.id}/metastore"  # Specify the cloud storage path
  owner         = "uc admins"  # Set the owner (username/groupname/sp application_id)
  region        = "us-east-1"  # Specify the region
  force_destroy = true  # Enable force destroy if needed
}

In this example:

  • name: Provide a unique name for your metastore.
  • storage_root: Define the path on your cloud storage account where managed Databricks tables are stored. If no storage_root is defined for the metastore, each catalog must have its own storage root.
  • owner: Specify the owner of the metastore (e.g., user, group, or application ID).
  • region: Set the region where the metastore resides.
  • force_destroy: Enable this option if you want to allow forceful destruction of the resource.

Once you’ve created the metastore, you can reference it in another Terraform project by using its ID. For example, if you have a separate Terraform configuration where you need to use this metastore, you can retrieve its ID and use it as follows:

resource "databricks_metastore_assignment" "this" {
  metastore_id = databricks_metastore.metastore.id
  workspace_id  = local.workspace_id  # Specify the workspace ID where you want to assign the metastore
}

Replace local.workspace_id with the actual workspace ID where you want to use this metastore. This assignment ensures that the metastore is associated with the appropriate Databricks workspace1.

Remember to adjust the configuration according to your specific cloud provider (AWS, Azure, or GCP) and workspace requirements. If you encounter any issues or need further assistance, feel free to ask!

 

ducng
New Contributor II

Hi @Kaniz

Thank you for the prompt response, not sure why my post didn't show fully

What I hope to achieve is to access the UC metastore ID programmatically instead of embedding it to another repo manually using local variable.

Thank you for your time! 🙂

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!