cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to query existing storage and network configuration (AWS)?

ThisNoName
New Contributor III

Trying to provision a simple workspace.  All the code I can find look something like the following, where credential/storage/network resources are created, then referenced. 

In my case, it's a separate repo and try to reuse existing configurations.  I was able to query mws_credential and find credential_id, but there are no data source for storage_configuration and mws_networks, according to databricks documentation.

Any hints where I should look?  Undocumented data sources?  Wrapped up somewhere in other sources?  I can see both configuration_id and network_id in tfstate but that's probably a bit too much

Thanks

data "databricks_mws_credentials" "all" {
  provider = databricks.mws
}
resource "databricks_mws_workspaces" "this" {
  provider                 = databricks.mws
  credentials_id           = databricks_mws_credentials.this.credentials_id
  storage_configuration_id = databricks_mws_storage_configurations.this.storage_configuration_id
  network_id               = databricks_mws_networks.this.network_id
  ... ...
}

resource "databricks_mws_credentials" "this" {
  provider         = databricks.mws
  ... ...
}

resource "databricks_mws_storage_configurations" "this" {
  provider                   = databricks.mws
  ... ...
}

resource "databricks_mws_networks" "this" {
  provider           = databricks.mws
  security_group_ids = var.security_group_ids
  subnet_ids         = var.vpc_private_subnets
  vpc_id             = var.vpc_id
  ... ...
}

 

1 ACCEPTED SOLUTION

Accepted Solutions

raphaelblg
Databricks Employee
Databricks Employee

Hi @ThisNoName

Based on your description it looks like you're trying to get Databricks account level information for networks and storage configurations. You can easily achieve that through the Databricks account API

Here are the docs:

- Get all network configurations: https://docs.databricks.com/api/account/networks/list

- Get all storage configurations: https://docs.databricks.com/api/account/storage/list

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks

View solution in original post

2 REPLIES 2

raphaelblg
Databricks Employee
Databricks Employee

Hi @ThisNoName

Based on your description it looks like you're trying to get Databricks account level information for networks and storage configurations. You can easily achieve that through the Databricks account API

Here are the docs:

- Get all network configurations: https://docs.databricks.com/api/account/networks/list

- Get all storage configurations: https://docs.databricks.com/api/account/storage/list

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks

So there's no official terraform data sources for these?  I guess maybe curl and grep or something?  Beats fumbling through tfstate. 

Will give a try.  Thanks.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group