Updating Databricks SQL Warehouse using Terraform

[Sudheer2]

We can Update SQL Warehouse manually in Databricks.

Click SQL Warehouses in the sidebar

In Advanced options

We can find Unity Catalog toggle button there!

While Updating Existing SQL Warehouse in Azure to enable unity catalog using terraform, I couldn't find proper documentation.

Can anyone please explain.

#SqlWarehouse #EnableUnityCatalog #Compute #terraform

[Kaniz_Fatma]

Hi @Sudheer2,

• The Unity Catalog is a feature in Databricks SQL Warehouse that allows you to query data across multiple databases and tables seamlessly. It provides a unified view of your data.
• When you enable the Unity Catalog, you can access tables from different databases within the same workspace without explicitly specifying the database name.
• To update the configuration of a SQL warehouse in Databricks, you can use the Databricks REST API.
• Specifically, you can use the POST /api/2.0/sql/warehouses/{id}/edit endpoint to modify the configur...¹.
• The {id} parameter represents the ID of the warehouse you want to configure.
• While Terraform is a powerful infrastructure-as-code tool, it’s essential to note that it primarily focuses on managing cloud resources (such as VMs, networks, storage, etc.).
• For managing the configuration of services within Databricks (including SQL Warehouses), you might need to combine Terraform with other tools or manual steps.
• The recommended pattern for updating an Azure SQL Data Warehouse table from Databricks is as follows:
  • Use the Databricks Azure SQL DW Connector to bulk load data into a staging table.
  • Alternatively, write the new data to files in blob storage or a data lake.
  • Create a Polybase External Table in Azure SQL DW to hold the new rows

 Good luck with your SQL Warehouse configuration! 😊🚀³⁴².

1.  

[KaseyHampton]

Thanks for answering, you made my day. If I want to know more, I will ask by starting my own thread.