error creating token when creating databricks_mws_workspace resource on GCP

[yurib]

resource "databricks_mws_workspaces" "this" {
  depends_on = [ databricks_mws_networks.this ]
  provider = databricks.account
  account_id = var.databricks_account_id
  workspace_name = "${local.prefix}-dbx-ws"
  location = var.google_region

  cloud_resource_container {
    gcp {
      project_id = var.google_project
    }
  }
  private_access_settings_id = var.databricks_pas_id 
  network_id = databricks_mws_networks.this.network_id
  gke_config {
    connectivity_type = "PRIVATE_NODE_PUBLIC_MASTER"
    master_ip_range = var.mws_workspace_gke_master_ip_range
  }
  token {}
  pricing_tier = "PREMIUM"
}
...
│ Error: cannot create mws workspaces: cannot create token: failed during request visitor: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://xxxx.gcp.databricks.com, google_service_account=xxx​

The workspace appears to be created - I can interact with it in the Databricks account console and with the respective google resources via google cloud console, but the `terraform apply` command fails with the above error.

`terraform destroy` fails with a similar error about reading the token.

The errors go away if I authenticate with the newly created workspace (databricks auth login --host https://xxx.gcp.databricks.com  ) but at that point terraform marks the workspace resource as tainted, forcing re-creating it and producing the same error for the yet newer workspace.

versions / provider definition:

terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = "1.48.3"
      configuration_aliases = [ databricks.account, databricks.workspace ]
    }
  }
}

provider "databricks" {
  alias                  = "account"
  host                   = var.account_console_url
  account_id             = var.databricks_account_id
  google_service_account = var.databricks_google_service_account
}

provider "databricks" {
  alias                  = "workspace"
  host                   = databricks_mws_workspaces.this.workspace_url
  token                  = databricks_mws_workspaces.this.token[0].token_value
}

[yurib · Accepted solution]

my issue was caused be credentials in `~/.databrickscfg` (generated by databricks cli) taking precedence over the creds set by `gcloud auth application-default login`. google's application default creds should be used when using the databricks google service account for deploying resources. deleting / renaming ~/.databrickscfg resolved my problem.