cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

On Demand Pool Configuration & Policy definition

n1399
New Contributor II

I'm using Job cluster and created compute policies for library management and now I'm trying to use pools in databricks. I'm getting error like this : 

Cluster validation error: Validation failed for azure_attributes.spot_bid_max_price from pool, the value must be present

here is the policy definition in cluster policy 

"azure_attributes.spot_bid_max_price": {
    "type": "unlimited",
    "defaultValue": 100,
    "hidden": false
  },

 

This definition is already inherited in the policy definition, I'm unable to remove them

 

1 REPLY 1

LRALVA
Honored Contributor

@n1399 

The error "Validation failed for azure_attributes.spot_bid_max_price from pool, the value must be present" suggests that the spot bid max price is required, but it’s either missing or not correctly inherited from the compute policy when using pools.

Possible Solutions:

1.  Explicitly Define spot_bid_max_price in Pool Configuration

Since the pool inherits cluster policies but still requires a valid spot_bid_max_price , ensure that:

  • The instance pool has an explicit spot_bid_max_price set.
  • Navigate to Compute → Instance Pools, edit the pool, and check if  spot_bid_max_price is set.

If not:

  • Set spot_bid_max_price = 100 explicitly in the pool configuration.

2. 

Overide spot_bid_max_pricein Cluster Policy (Allow Specific Values)

Modify the cluster policy to explicitly allow a set range instead of unlimited:

"azure_attributes.spot_bid_max_price": {
"type": "fixed",
"value": 100
}

 

or if you need flexibility:

"azure_attributes.spot_bid_max_price": {
"type": "range",
"minValue": 0,
"maxValue": 100
}

3. Remove spot_bid_max_price From Pool and Use Default Values

If the policy is inherited but causing conflicts, try removing it from the pool definition and letting the cluster policy enforce it instead:

  • Go to Compute → Instance Pools.
  • Remove any direct reference to spot_bid_max_price in the pool settings.

Then, create a cluster using the pool and check if the policy correctly enforces the value.

4.  Check Databricks Runtime Version & Policy Restrictions

Some older Databricks runtime versions might not fully support inherited policies from pools. If you’re using an older version:

  • Upgrade to a recent Databricks Runtime (e.g., 11.x or later).
  • Try using a policy without hidden attributes first to debug any constraints.
LR