Errors on Databricks and Terraform upgrades for DABS

[SumeshD]

Hi All

I've recently upgraded my Databricks CLI to 0.235 and Terraform provider to 1.58 locally on my machine and my DABs deployments have broken. They did work in the past with previous versions and now I can't even run Terraform -v. The command Databricks Bundle validate still works but when I try to run Databricks Bundle Deploy I get the error code block below.

I've tried looking for the elements that seem to be in conflict but cannot find anything in my config code that has "driver_instance_pool_id" defined.

If anyone can help or point me in the right direction I would greatly appreciate it.

Error message:

2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_instance_pool_id job_cluster.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_instance_pool_id job_cluster.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_node_type_id job_cluster.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.azure_attributes job_cluster.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.aws_attributes job_cluster.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.aws_attributes job_cluster.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_node_type_id job_cluster.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [TRACE] over recursion limit, skipping field: jobs.ForEachTask, max depth: 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_instance_pool_id task.0.for_each_task.0.task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_instance_pool_id task.0.for_each_task.0.task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_node_type_id task.0.for_each_task.0.task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.azure_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_node_type_id task.0.for_each_task.0.task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_instance_pool_id task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_instance_pool_id task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_node_type_id task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.azure_attributes task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.aws_attributes task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.aws_attributes task.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_node_type_id task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
Databricks Terraform Provider

Version 1.58.0

https://registry.terraform.io/providers/databricks/databricks/latest/docs

This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically

Thanks 

Sumesh

[arielhumana]

having same issue here when running the terraform binary plugin deployed byt DABs.

[Alberto_Umana]

Hi @SumeshD,

Can you run databricks bundle debug terraform to obtain more details on the failure?

The error messages you are encountering, such as "ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1," indicate that there are conflicts in your Terraform configuration related to the new_cluster attributes. These conflicts are being skipped because the TypeList block has MaxItems not equal to 1