mark_ott
Databricks Employee
Databricks Employee

Here are workarounds you can use until a fix is provided:

  • Deploy Jobs Separately: Deploy the first job using DAB, then uncomment and deploy the second one as you found works. This avoids the conflict encountered when both are deployed together.

  • Force Resource Recreation: Use flags or settings (such as --force deploy or explicit destroy-and-redeploy) to trigger full resource recreation, which sometimes sidesteps YAML parsing bugs.

  • Check for DAB Updates: Monitor DAB releases and upgrade promptly, since Databricks is actively patching bundle logic, especially around cluster resource diffing in multi-job bundles.

  • Consistent EBS Specification: Explicitly specify identical EBS config blocks for both driver and worker nodes, even if redundant, to avoid edge case parsing issues (though YAML should not require this, it can help with bugs).

  • Manual Management: As last resort, create job clusters manually via GUI for jobs that hit this error, then use DAB only for jobs that do not trigger the bug.

  • Simplified Removal: Temporarily switch your jobs.yml to use only worker node types for both driver and worker, if possible, then revert to your preferred config once the bug is patched.