I was working on a proof of concept (POC) using the assert bundle. My job configuration in the .yml file worked yesterday, but it threw an error today during a demo to the team.
The error was likely due to an update to the Databricks API. After some trial and error and reviewing the API documentation, Adding the following to the job configuration resolved the issue:
kind: CLASSIC_PREVIEW
How can we anticipate such updates in the future and prevent these issues?
Here is the error message:
Error: terraform apply: exit status 1
Error: cannot update job: data_security_mode: DATA_SECURITY_MODE_DEDICATED is not allowed with unspecified kind.
with databricks_job.sample_python_job,
on bundle.tf.json line 69, in resource.databricks_job.sample_python_job:
69:
