Persisting and managing tables and table schemas in Unity Catalog
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-16-2024 02:38 AM
Hello Databricks Community,
we are currently looking for a way to persist and manage our unity catalog tables in an IaC manner. That is, we want to trace any changes to a table's schema and properties and ideally be able to roll back those changes seamlessly. Using the databricks_sql_table terraform resource does not seem to be the ideal solution since "changing the column definitions for a table will require dropping and re-creating the table" (https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/sql_table). Also in this issue (https://github.com/databricks/terraform-provider-databricks/issues/1624) it is mentioned that using the API and terraform is discouraged.
Is there a better alternative to managing tables? We were thinking of having notebooks with DDL scripts alongside our applications that define the tables.