- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-29-2025 04:21 PM
I am seeing some legacy hive_metastore corruption (especially tables created as parquet instead of Delta) lately in my client's place, who is in the midst of migrating to UC. We were provided with a Scala code to remove the erroneous Parquet files physically. Anyone facing a similar issue?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-29-2025 04:47 PM
HI @thbeh_com
Yes, this is a fairly common issue during UC migrations, especially with legacy Hive metastore tables. The corruption typically happens because:
- Metadata-data misalignment - Hive metastore references files that no longer exist or have been moved
- Parquet schema evolution issues - Column changes not properly reflected in metastore
- Concurrent operations during migration causing inconsistent states
- File system operations bypassing Hive metastore updates
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-29-2025 05:05 PM
