Cluster policy not showing while creating delta live table pipeline

[Mbinyala]

Hi all!!!

I have created a cluster policy but when i want to use that while creating dlt pipeline, It is showing none. I have checked I have all the necessary permissions to create cluster policies. Still, in dlt ui it is showing none.

[Rishitha]

Hi, even i'm facing the same issue. I have a policy with all permissions and it is still not showing up in the DLT pipeline setting UI to select.  Please help

[btafur]

You can check your cluster policy. There is a property called "cluster_type" where you can specify if it will apply to "dlt" or not. https://docs.databricks.com/en/administration-guide/clusters/policies.html

  "cluster_type": {
    "type": "fixed",
    "value": "dlt"
  },

[Rishitha]

@btafur  That helped, the issue is fixed now. Thanks. 

a follow up question, can we give 2 values there for cluster type, something like this

"cluster_type": {
    "type": "fixed",
    "value": "dlt"
  },
"cluster_type": { 
    "type": "fixed", 
    "value": "job" 
}

or comma separated values.

Thanks again!

[btafur]

You should be able to use "allowlist" instead of "fixed" for multiple values. For example:

 "cluster_type": {
  "type": "allowlist",
  "defaultValue": "all-purpose",
  "values": [
   "all-purpose",
   "job",
   "dlt"
  ]
 },

[Rishitha]

That's perfect thank you. @btafur 

[Rishitha]

@btafur  Can we also set the auto_terminate minutes with the policy?  (for the dlt cluster type)