topic Re: On Demand Pool Configuration & Policy definition in Get Started Discussions

On Demand Pool Configuration & Policy definition

n1399 — Sun, 09 Mar 2025 05:03:39 GMT

I'm using Job cluster and created compute policies for library management and now I'm trying to use pools in databricks. I'm getting error like this :

Cluster validation error: Validation failed for azure_attributes.spot_bid_max_price from pool, the value must be present

here is the policy definition in cluster policy

"azure_attributes.spot_bid_max_price": {
    "type": "unlimited",
    "defaultValue": 100,
    "hidden": false
},

This definition is already inherited in the policy definition, I'm unable to remove them

Re: On Demand Pool Configuration & Policy definition

lingareddy_Alva — Sun, 09 Mar 2025 17:58:12 GMT

@n1399

The error "Validation failed for azure_attributes.spot_bid_max_price from pool, the value must be present" suggests that the spot bid max price is required, but it’s either missing or not correctly inherited from the compute policy when using pools.

Possible Solutions:

1. Explicitly Define spot_bid_max_price in Pool Configuration

Since the pool inherits cluster policies but still requires a valid spot_bid_max_price , ensure that:

The instance pool has an explicit spot_bid_max_price set.
Navigate to Compute → Instance Pools, edit the pool, and check if spot_bid_max_price is set.

If not:

Set spot_bid_max_price = 100 explicitly in the pool configuration.

Overide spot_bid_max_pricein Cluster Policy (Allow Specific Values)

Modify the cluster policy to explicitly allow a set range instead of unlimited:

"azure_attributes.spot_bid_max_price": {
"type": "fixed",
"value": 100
}

or if you need flexibility:

"azure_attributes.spot_bid_max_price": {
"type": "range",
"minValue": 0,
"maxValue": 100
}

3. Remove spot_bid_max_price From Pool and Use Default Values

If the policy is inherited but causing conflicts, try removing it from the pool definition and letting the cluster policy enforce it instead:

Go to Compute → Instance Pools.
Remove any direct reference to spot_bid_max_price in the pool settings.

Then, create a cluster using the pool and check if the policy correctly enforces the value.

4. Check Databricks Runtime Version & Policy Restrictions

Some older Databricks runtime versions might not fully support inherited policies from pools. If you’re using an older version:

Upgrade to a recent Databricks Runtime (e.g., 11.x or later).
Try using a policy without hidden attributes first to debug any constraints.

Re: On Demand Pool Configuration & Policy definition

Poorva21 — Sat, 06 Dec 2025 16:45:53 GMT

This error occurs because instance pools require a concrete spot bid max price value, even if the cluster policy marks it as unlimited. Set an explicit value (e.g., 100) directly in the instance pool configuration, or switch the pool to on-demand nodes to avoid spot validation entirely. The policy alone cannot satisfy this requirement.