topic Unity Catalog is missing column in Catalog Explorer in Data Engineering

Unity Catalog is missing column in Catalog Explorer

chad_woodhead — Thu, 25 Jan 2024 22:14:20 GMT

I have just altered one of my tables and added a column.

ALTER TABLE tpch.customer ADD COLUMN C_CUSTDETAILS struct<key:string,another_key:string,boolean_key:boolean,extra_key:string,int_key:long,nested_object:struct<more:long,arrayOne:array<string>>>

As you can see this new column is of type Struct.

When I look at the table's DDL via a SHOW CREATE TABLE or DESCRIBE TABLE I can see the newly added column.

However, when I look at the table in Catalog Explorer in Unity Catalog the new column is not showing.

If I try to run the ADD COLUMN command again I get an error that the column exists.

I have tried running a REPAIR TABLE tpch.customer SYNC METADATA and that has not helped.

Does anyone know what might be going on and how to resolve this?

Re: Unity Catalog is missing column in Catalog Explorer

LaurentLeturgez — Thu, 01 Feb 2024 15:58:26 GMT

Hi @chad_woodhead

Weird behavior as I tried to reproduce this into my environment on a delta external table as well as on a managed table. And I'm not able to reproduce what you encountered.

Which DBR version are you using ? I used the latest DBR 14.3 LTS to make my tests. What is the file format used (is it delta ?)

In the meantime, I suggest to raise a ticket to our support as there will be a second investigation track opened.

HTH.

Laurent

Re: Unity Catalog is missing column in Catalog Explorer

smike — Tue, 19 Nov 2024 08:29:50 GMT

Hi, did you solve this issue? I encountered the exact same problem as well.

Re: Unity Catalog is missing column in Catalog Explorer

Vinodt — Wed, 22 Jan 2025 17:12:40 GMT

I have the same issue anyone find a fix? I use DBR 14.3. The external table hosted on hive metastore has no issue while the UC version doesn't. Note: The issue doesn't happen for every table in the catalog

Re: Unity Catalog is missing column in Catalog Explorer

Lakshay — Tue, 28 Jan 2025 09:05:16 GMT

Can you share the steps for reproducing this issue? If it is reproducible, I can investigate.

Re: Unity Catalog is missing column in Catalog Explorer

GoToJDenman — Tue, 02 Dec 2025 20:46:39 GMT

I had this error just recently. I did basically the same table transformation 4 times over the course of 2 days. Added two new fields to two different tables using the same SQL syntax. It worked 3 out of 4 times, but for 1 the column is not available in the catalog- despite being there when I describe the table in a notebook.