Errors on Databricks and Terraform upgrades for DABS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-28-2024 10:25 PM
Hi All
I've recently upgraded my Databricks CLI to 0.235 and Terraform provider to 1.58 locally on my machine and my DABs deployments have broken. They did work in the past with previous versions and now I can't even run Terraform -v. The command Databricks Bundle validate still works but when I try to run Databricks Bundle Deploy I get the error code block below.
I've tried looking for the elements that seem to be in conflict but cannot find anything in my config code that has "driver_instance_pool_id" defined.
If anyone can help or point me in the right direction I would greatly appreciate it.
Error message:
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_instance_pool_id job_cluster.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_instance_pool_id job_cluster.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_node_type_id job_cluster.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.azure_attributes job_cluster.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.aws_attributes job_cluster.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.aws_attributes job_cluster.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [job_cluster.0.new_cluster.0.driver_node_type_id job_cluster.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [TRACE] over recursion limit, skipping field: jobs.ForEachTask, max depth: 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_instance_pool_id task.0.for_each_task.0.task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_instance_pool_id task.0.for_each_task.0.task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_node_type_id task.0.for_each_task.0.task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.azure_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.driver_node_type_id task.0.for_each_task.0.task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_instance_pool_id task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_instance_pool_id task.0.new_cluster.0.instance_pool_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_node_type_id task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.azure_attributes task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.aws_attributes task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.aws_attributes task.0.new_cluster.0.azure_attributes], path contains TypeList block with MaxItems not equal to 1
2024/11/29 16:13:46 [DEBUG] ConflictsWith skipped for [task.0.new_cluster.0.driver_node_type_id task.0.new_cluster.0.node_type_id], path contains TypeList block with MaxItems not equal to 1
Databricks Terraform Provider
Version 1.58.0
https://registry.terraform.io/providers/databricks/databricks/latest/docs
This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically
Thanks
Sumesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-24-2024 07:14 AM
having same issue here when running the terraform binary plugin deployed byt DABs.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-24-2024 07:59 AM
Hi @SumeshD,
Can you run databricks bundle debug terraform to obtain more details on the failure?
The error messages you are encountering, such as "ConflictsWith skipped for [task.0.for_each_task.0.task.0.new_cluster.0.aws_attributes task.0.for_each_task.0.task.0.new_cluster.0.gcp_attributes], path contains TypeList block with MaxItems not equal to 1," indicate that there are conflicts in your Terraform configuration related to the new_cluster
attributes. These conflicts are being skipped because the TypeList
block has MaxItems
not equal to 1

