Create workspace in Databricks deployed in Google Cloud using terraform

[horatiug]

In the documentation

https://registry.terraform.io/providers/databricks/databricks/latest/docs

https://docs.gcp.databricks.com/dev-tools/terraform/index.html

I could not find documentation on how to provision Databricks workspaces in GCP. Only creating Databricks workspaces in AWS and Azure is mentioned.

Is it possible to provision Databricks workspaces using terraform in Google Cloud Platform ?

[Debayan]

Hi, @horatiu guja​ , as of now we do not have public document on the same. It is possible, but, there is no straight forward way to do it.

[Debayan]

In addition to the previous comment, attaching the document which is in private preview as of now.

Please let us know if you need further clarification on the same.

[horatiug]

Hi @Debayan Mukherjee​ 

thanks for the document. Really valuable resource. I have tried and got blocked into following error:

databricks_mws_workspaces.this: Creating...

╷

│ Error: cannot create mws workspaces: BAD_REQUEST: workspace creation REST API is not available to use. Please contact Databricks support

│ 

│  with databricks_mws_workspaces.this,

│  on main.tf line 35, in resource "databricks_mws_workspaces" "this":

│  35: resource "databricks_mws_workspaces" "this" {

│ 

Any reason why this could happen ? Is this API maybe working from Databricks PREMIUM plan and not working in STANDARD plan ?

Thanks

Horatiu

[Debayan]

Hi @horatiu guja​ , let me check on the error. This is still under development and does not work any many cases (hence the document is not published publicly).

[Debayan]

@horatiu guja​ , This feature works only for premium plan.

[horatiug]

Hi @Debayan Mukherjee​ 

following the document I was able to create workspace and add some infrastructure in the workspace. Issue I face now is when running terraform destroy. Then I get following error:

╷

│ Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_credentials, google_service_account. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

│ 

│  with databricks_user.me,

│  on main.tf line 70, in resource "databricks_user" "me":

│  70: resource "databricks_user" "me" {

│ 

When the error occurs I am still logged in GCP using gcloud auth application-default login. Any hint what could be the reason for that ?

thanks

Horatiu

[Anonymous]

Hi @horatiu guja​ 

Does @Debayan Mukherjee​ response answer your question?

If yes, would you be happy to mark it as best so that other members can find the solution more quickly? Else, we can help you with more details.

[horatiug]

Hi @Vidula Khanna​ 

solution works and will mark it as best. Just want to mention that Accounts API are needed and in GCP this is in preview. To use them the preview must be enabled for the corresponding account.

reggards

Horatiu