- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-13-2022 01:52 AM
I am looking into Unity too. I'd say it is both great and limited, but more limited than great.
Great because you have some very interesting features like column/row-based access, and lineage.
But it is still very limited because on the heavy focus on tables and delta lake.
They kinda seem to have forgotten that A LOT of data still resides in common parquet files.
f.e.
Overwrite mode for DataFrame write operations into Unity Catalog is supported only for Delta tables, not for other file formats.
This alone makes me wonder if we should use it, or look into DataHub/Amundsen.
It is still a new product, so new features might get added, but right now I probably will not be using it.