Cluster policy not showing while creating delta live table pipeline
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-15-2023 03:49 AM
Hi all!!!
I have created a cluster policy but when i want to use that while creating dlt pipeline, It is showing none. I have checked I have all the necessary permissions to create cluster policies. Still, in dlt ui it is showing none.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 11:20 AM
Hi, even i'm facing the same issue. I have a policy with all permissions and it is still not showing up in the DLT pipeline setting UI to select. Please help
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 12:31 PM - edited 09-22-2023 12:31 PM
You can check your cluster policy. There is a property called "cluster_type" where you can specify if it will apply to "dlt" or not. https://docs.databricks.com/en/administration-guide/clusters/policies.html
"cluster_type": {
"type": "fixed",
"value": "dlt"
},
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 12:39 PM
@btafur That helped, the issue is fixed now. Thanks.
a follow up question, can we give 2 values there for cluster type, something like this
"cluster_type": {
"type": "fixed",
"value": "dlt"
},
"cluster_type": {
"type": "fixed",
"value": "job"
}
or comma separated values.
Thanks again!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 12:44 PM
You should be able to use "allowlist" instead of "fixed" for multiple values. For example:
"cluster_type": { "type": "allowlist", "defaultValue": "all-purpose", "values": [ "all-purpose", "job", "dlt" ] },
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 12:46 PM
That's perfect thank you. @btafur
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2023 12:53 PM
@btafur Can we also set the auto_terminate minutes with the policy? (for the dlt cluster type)

