Accessing Unity Catalog Externally (Outside of Databricks) as Hive Metastore
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-24-2024 06:27 PM
Hello,
I'm designing a system that combines a lake-house and a real-time path that federates across multiple databases. The requirements led us to adopt Starburst Galaxy (a cloud-hosted distributed query engine) for query federation across the various DBs and (mostly reading) into the lake-house. However, I would also like to make use of the Databricks platform to manage the lake-house and upcoming ML work. The tentative approach to fuse these two platform: let Databricks run in the typical setup (bronze-silve-gold, Delta Lake / DLT, etc.), and let Galaxy / Trino access the Unity Catalog somehow so they both see the same lake metadata and data (Delta Lake tables).
Question:
Is this (accessing Unity Catalog as Hive Metastore externally) feasible? The idea was based on this blog post: https://www.databricks.com/blog/extending-databricks-unity-catalog-open-apache-hive-metastore-api. Any additional pointers on this matter is greatly appreciated.
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-30-2024 07:42 AM
Thank you @Retired_mod.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-12-2024 04:41 PM
Hello there!
So any updates so far? Also interesting in this solution.
![](/skins/images/97567C72181EBE789E1F0FD869E4C89B/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/97567C72181EBE789E1F0FD869E4C89B/responsive_peak/images/icon_anonymous_message.png)