nayan_wylde
Esteemed Contributor II

This is a real, known class of behavior with Unity Catalog (UC), and what you observed does not point to data corruption. It does point to metadata visibility, caching, and catalog context interactions—especially with serverless compute and spark.catalog.tableExists().

Nothing was “corrupted.”
Nothing was permanently unstable.
The behavior you saw is consistent with Unity Catalog metadata caching + catalog context + serverless/SQL path differences.
spark.catalog.tableExists() is not a strong guarantee under UC, especially right after DDL changes.
The fact that DROP TABLE started behaving “normally” again strongly indicates cache invalidation / propagation timing, not corruption.