The issue regarding renaming a Unity Catalog using Terraform is rooted in the way the Terraform databricks_catalog
resource is implemented. Specifically, the catalog name is being used as the resource's ID, which is a direct consequence of limitations in the Databricks REST API response that does not include an actual catalog ID. Because of this:
- It is currently not possible to rename a catalog via Terraform.
- Renaming a catalog outside Terraform (e.g., via the UI) causes Terraform to interpret the catalog as deleted and recreate it based on the Terraform configuration, leading to duplicate catalogs.
From your investigation, it seems these behaviors align with certain known limitations within the Databricks Terraform provider. The relevant code snippet in resource_catalog.go
confirms the use of the name as the ID:
Id: catalog.Name,
.
You have correctly identified the root cause: the Databricks REST API does not provide a unique catalog ID in its response for workspace catalog operations. Although the Terraform provider relies on the API, this limitation is outside the scope of Terraform itself.
Hope this helps, Lou.