cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Override default Personal Compute policy using terraform / disable Personal Compute policy

ossinova
Contributor II

I want to programmatically do some adjustments to the default personal compute resource or preferably create my own custom one based on the same configuration or policy family (in which all users can gain access to) when deploying a new workspace using terraform. I know an account admin can enable Manage the Personal Compute policy - Azure Databricks | Microsoft Learn at an account level although I am uncertain if this can be applied for other policies too (for instance the custom one) so that the default is disabled and the custom one takes over. 

Is there a way to override the defualt personal compute policy? I need the following to be overridden:

"autotermination_minutes": {
"type": "fixed",
"value": 30
}

 

This auto termination should be the default for all personal compute clusters in the workspace. Is there a way to make this work without having to manually adjust it post-deployment?

3 REPLIES 3

feiyun0112
Honored Contributor

use api to create new cluster, set autotermination_minutes parameter

https://docs.databricks.com/api/workspace/clusters/create#autotermination_minutes 

I am aware of the API. However, that is out of scope of the question as I want to configure a policy upon workspace creation using Terraform as opposed to creating a cluster. The policy should override the default configuration of the current default one shipped with databricks. 

jsimonovic
New Contributor II

Hi, I think this doc might be useful: Override default Personal Compute policy using ter... - Databricks - 61695, basically is an override process by creating a databricks_cluster_policy resource:

 

locals {
  personal_vm_override = {
    "autotermination_minutes" : {
      "type" : "fixed",
      "value" : 220,
      "hidden" : true
    },
    "custom_tags.Team" : {
      "type" : "fixed",
      "value" : var.team
    }
  }
}

resource "databricks_cluster_policy" "personal_vm" {
  policy_family_id                   = "personal-vm"
  policy_family_definition_overrides = jsonencode(personal_vm_override)
  name                               = "Personal Compute"
}

 

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group