For a long time, one of the hardest questions in lakehouse architecture was:
How do we let external engines access governed data without bypassing governance?
Databricks is making this pattern much cleaner with Unity Catalog external access.
The idea is simple but powerful: keep Unity Catalog as the governance layer, while allowing external systems to interact with Databricks-managed data through supported open interfaces like the Unity REST API and Apache Iceberg REST catalog or cloud URIs.
What I find especially interesting is credential vending. Instead of giving external systems broad, long-lived cloud credentials, Unity Catalog can issue short-lived credentials scoped to a table or path. Those credentials inherit the privileges of the Databricks principal used for the integration.
That has big implications:
- Centralized governance through Unity Catalog
- Less direct credential sharing
- Better alignment between Databricks permissions and external access
- More realistic multi-engine lakehouse architectures
- Support for both Delta and Iceberg access patterns
There are still important caveats. Some features are in Preview or Beta, not every client supports credential vending, and direct access to cloud storage still requires careful governance and network controls. Databricks also warns about consistency and corruption risks when multiple external clients write to the same Delta table in S3.
The lakehouse is becoming less about “all workloads must run in one engine” and more about one governed data layer, many interoperable engines.
