Databricks Asset Bundles fails deploy but works on the GUI with same parameters

[LeoRickli]

I'm running into an issue when running databricks bundle deploy when using job clusters.

When I run databricks bundle deploy on a new workspace or after destroying previous resources, the deployment fails with the error: Error: cannot update job: At least one EBS volume must be attached for clusters created with node type m8g.xlarge.

This error occurs even though the cluster configuration in my jobs.yml file correctly specifies EBS volumes under aws_attributes, as shown below.

- job_cluster_key: process_to_bronze_cluster
  new_cluster:
    spark_version: 17.0.x-scala2.13
    aws_attributes:
      first_on_demand: 1
      availability: SPOT_WITH_FALLBACK
      instance_profile_arn: <instance_profile>
      spot_bid_price_percent: 100
      ebs_volume_type: GENERAL_PURPOSE_SSD
      ebs_volume_count: 3
      ebs_volume_size: 100
    node_type_id: m8g.large
    driver_node_type_id: m8g.xlarge # This in particular is triggering the error
    data_security_mode: SINGLE_USER
    runtime_engine: PHOTON
    autoscale:
      min_workers: 1
      max_workers: 4

I don't think it's an issue isolated with the node type but rather a but where it's not detecting the EBS volume attached to this node type. But we have two weird things:

1. I copied this job as a yaml from the Databricks GUI and it's working there fine.
2. This is my second job inside the jobs.yml file. If I comment-out this job and use databricks bundle deploy it will deploy only the first job. Then if I uncomment the second job and deploy, it will work!

So these two factors are clearly learning towards a bug on DABs, right? What is the workaround that I could do so I wait for a fix on this one?

Thanks!

[mark_ott]

Here are workarounds you can use until a fix is provided:

• Deploy Jobs Separately: Deploy the first job using DAB, then uncomment and deploy the second one as you found works. This avoids the conflict encountered when both are deployed together.

• Force Resource Recreation: Use flags or settings (such as --force deploy or explicit destroy-and-redeploy) to trigger full resource recreation, which sometimes sidesteps YAML parsing bugs.

• Check for DAB Updates: Monitor DAB releases and upgrade promptly, since Databricks is actively patching bundle logic, especially around cluster resource diffing in multi-job bundles.

• Consistent EBS Specification: Explicitly specify identical EBS config blocks for both driver and worker nodes, even if redundant, to avoid edge case parsing issues (though YAML should not require this, it can help with bugs).

• Manual Management: As last resort, create job clusters manually via GUI for jobs that hit this error, then use DAB only for jobs that do not trigger the bug.

• Simplified Removal: Temporarily switch your jobs.yml to use only worker node types for both driver and worker, if possible, then revert to your preferred config once the bug is patched.

[NandiniN]

Hello @LeoRickli 

Are you setting apply_policy_default_values?

https://docs.databricks.com/en/administration-guide/clusters/policies.html#:~:text=Default%20values%....

After you update a policy's definitions, the compute resources created using that policy aren't automatically updated with the new policy definitions. You can choose to update all or some of these compute resources using policy compliance enforcement. See Enforce policy compliance.

Thanks!