When adding new columns with overwriteSchema = True it does not add it to UC

ossinova
Contributor II

‎10-14-2022 03:12 AM

I have discovered some strange behavior when adding new columns to a table associated with Unity Catalog. I overwrote the data specifying overwriteSchema = True and passing in the abfss path as it is an external table.

When quering against it, it is discovered, however when I use information_schema and data explorer it is not listed.

Screenshot 2022-10-14 120710 

Does it have something to do with it still being associated to the old hive_metastore? I am having a hard time to figure it out and any help is appreciated.

Preview file
51 KB

Hubert-Dudek
Databricks MVP

‎10-20-2022 05:07 AM

I bet that issue is that you update it as an external table. So as it is not managed table, then infromation_schema is not updated.


My blog: https://databrickster.medium.com/
0 Kudos

Anonymous
Not applicable

‎11-27-2022 04:59 AM

Hi @Oscar Dyremyhr​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

0 Kudos