Unity Catalog is missing column in Catalog Explorer

chad_woodhead
New Contributor

‎01-25-2024 02:13 PM - edited ‎01-25-2024 02:14 PM

I have just altered one of my tables and added a column.

ALTER TABLE tpch.customer ADD COLUMN C_CUSTDETAILS struct<key:string,another_key:string,boolean_key:boolean,extra_key:string,int_key:long,nested_object:struct<more:long,arrayOne:array<string>>>

As you can see this new column is of type Struct.

When I look at the table's DDL via a SHOW CREATE TABLE or DESCRIBE TABLE I can see the newly added column.

chad_woodhead_0-1706220653227.png

However, when I look at the table in Catalog Explorer in Unity Catalog the new column is not showing.

chad_woodhead_1-1706220693600.png

If I try to run the ADD COLUMN command again I get an error that the column exists.

I have tried running a REPAIR TABLE tpch.customer SYNC METADATA and that has not helped.

Does anyone know what might be going on and how to resolve this?

0 Kudos

LaurentLeturgez
Databricks Employee
Databricks Employee

‎02-01-2024 07:58 AM

Hi @chad_woodhead 

Weird behavior as I tried to reproduce this into my environment on a delta external table as well as on a managed table. And I'm not able to reproduce what you encountered.

Which DBR version are you using ? I used the latest DBR 14.3 LTS to make my tests. What is the file format used (is it delta ?)

In the meantime, I suggest to raise a ticket to our support as there will be a second investigation track opened.

HTH.

Laurent

0 Kudos

smike
New Contributor II

‎11-19-2024 12:29 AM

Hi, did you solve this issue? I encountered the exact same problem as well. 

0 Kudos

Vinodt
New Contributor II

‎01-22-2025 09:09 AM - edited ‎01-22-2025 09:12 AM

I have the same issue anyone find a fix? I use DBR 14.3. The external table hosted on hive metastore has no issue while the UC version doesn't. Note: The issue doesn't happen for every table in the catalog 

0 Kudos

Lakshay
Databricks Employee
Databricks Employee

‎01-28-2025 01:05 AM

Can you share the steps for reproducing this issue? If it is reproducible, I can investigate.

0 Kudos

GoToJDenman
New Contributor II

‎12-02-2025 12:43 PM - edited ‎12-02-2025 12:46 PM

I had this error just recently. I did basically the same table transformation 4 times over the course of 2 days. Added two new fields to two different tables using the same SQL syntax. It worked 3 out of 4 times, but for 1 the column is not available in the catalog- despite being there when I describe the table in a notebook.

0 Kudos