Hello,

I am already recreating a new token at each time I init my Spark session. I do this using the Azure's oauth2 service to get a token lasting 1 hour and then using databricks API 2.0 to generate a new PAT.
And this code is working in local and in compute instances in Azure, but not Compute Clusters.

What I also tried: To generate a token in UI, working in local, then using it in my code in my compute cluster, and not working with the above error.

Cloud it be a network issue? I'm creating both compute instance/cluster in terraform:

resource "azurerm_machine_learning_compute_cluster" "cluster" {
  for_each = local.compute_cluster_configurations

  name     = each.key
  location = var.context.location

  vm_priority                   = each.value.vm_priority
  vm_size                       = each.value.vm_size
  machine_learning_workspace_id = module.mlw_01.id
  subnet_resource_id            = module.subnet_aml.id

  # AML-05
  ssh_public_access_enabled = false
  node_public_ip_enabled    = false

  identity {
    type = "UserAssigned"
    identity_ids = [
      azurerm_user_assigned_identity.compute_cluster_managed_identity.id
    ]
  }

  scale_settings {
    min_node_count                       = each.value.min_node_count
    max_node_count                       = each.value.max_node_count
    scale_down_nodes_after_idle_duration = each.value.scale_down_nodes_after_idle_duration
  }
}

# For each user, create a compute instance
resource "azurerm_machine_learning_compute_instance" "this" {
  for_each = local.all_users

  name                          = "${split("@", trimspace(local.all_users[each.key]["user_principal_name"]))[0]}-DS2-V2"
  location                      = var.context.location
  machine_learning_workspace_id = module.mlw_01.id
  virtual_machine_size          = "STANDARD_DS2_V2"
  identity {
    type = "UserAssigned"
    identity_ids = [
      azurerm_user_assigned_identity.this[each.key].id
    ]
  }
  assign_to_user {
    object_id = each.key
    tenant_id = var.tenant_id
  }
  node_public_ip_enabled = false
  subnet_resource_id     = module.subnet_aml.id
  description            = "Compute instance generated by Terraform for : ${local.all_users[each.key]["display_name"]} | ${local.all_users[each.key]["user_principal_name"]} | ${each.key} "
}

I'm using the same subnet, so it should react the same in network.