Asset Bundle API update issues

kanikeom
New Contributor II

‎03-18-2025 10:52 AM

I was working on a proof of concept (POC) using the assert bundle. My job configuration in the .yml file worked yesterday, but it threw an error today during a demo to the team.

The error was likely due to an update to the Databricks API. After some trial and error and reviewing the API documentation, Adding the following to the job configuration resolved the issue:
kind: CLASSIC_PREVIEW

How can we anticipate such updates in the future and prevent these issues?

Here is the error message:
Error: terraform apply: exit status 1
Error: cannot update job: data_security_mode: DATA_SECURITY_MODE_DEDICATED is not allowed with unspecified kind.
with databricks_job.sample_python_job,
on bundle.tf.json line 69, in resource.databricks_job.sample_python_job:
69: 

Johannes_E
New Contributor III

‎07-22-2025 10:17 PM

I've got the same issue and don't know how to fix it 😕

0 Kudos

mark_ott
Databricks Employee
Databricks Employee

‎11-10-2025 05:33 PM

Unexpected breaking changes to APIs—especially from cloud platforms like Databricks—can disrupt projects and demos. Proactively anticipating and rapidly adapting to such updates requires a combination of monitoring, process improvements, and technical safeguards.

Best Practices to Anticipate and Prevent API Breaking Changes

Monitor Official Sources

  • Subscribe to Databricks release notes and product update channels to learn about upcoming and recent changes.

  • Set up alerts from platforms such as GitHub repositories, provider changelogs (like Terraform Databricks provider), and Databricks community forums.

Pin Versions and Track Dependencies

  • Always pin specific versions of APIs, libraries, and providers (for example, the Databricks Terraform provider) in your configuration files. This helps ensure that your jobs use the known stable interface.

  • Regularly review your dependency version matrix and update intentionally, not automatically.

Test and Validate Regularly

  • Implement automated tests or pre-deployment checks using CI/CD pipelines to validate your configuration and code before every deployment.

  • Use staging environments that mirror production to test updates against the latest Databricks platform version.

Build Resilient Job Configurations

  • Specify all mandatory fields—such as kind: CLASSIC_PREVIEW—based on the latest API documentation so that your jobs do not rely on defaults that might change.

  • Track deprecation notices in API documentation and migrate off deprecated features early.

Engage with Databricks Support

  • Open support tickets or request clarifications when you encounter unexplained API behavior. Early feedback from support can guide your configuration changes.

Share Knowledge and Update Documentation

  • Document recent incidents, fixes (like adding kind: CLASSIC_PREVIEW), and lessons learned on internal wikis or playbooks.

  • Provide regular internal updates so your team is aware of how to resolve similar issues.

Error Cause Analysis

The error message you encountered:

Error: cannot update job: data_security_mode: DATA_SECURITY_MODE_DEDICATED is not allowed with unspecified kind.

shows Databricks changed the enforcement around the kind parameter when certain security modes are set. “Unspecified kind” is now invalid with some security settings; explicitly stating kind: CLASSIC_PREVIEW resolved the incompatibility.

Regularly checking for these sorts of changes to parameter requirements in the API documentation is essential to avoid future breakages.

Resources for Staying Up to Date

By combining active monitoring, disciplined dependency management, automated validation, and internal communication, your team can reduce the risk and impact of API updates causing POC or production failures.