- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-10-2023 08:07 AM
@Alex Davies :
Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region, and it is up to user permissions to restrict access appropriately.
One potential workaround for your use case could be to use separate Databricks accounts (with separate regions) for each environment (dev, staging, prod). Each Databricks account would have its own Unity Catalog and associated workspaces, and data would be segregated by account. This would allow you to enforce your policy of not allowing data to cross environment boundaries, but would also require managing separate Databricks accounts and associated resources for each environment.
Another option would be to use separate virtual networks (VNets) for each environment, and configure VNet peering between the VNets as appropriate to allow necessary communication between environments. This would allow you to further restrict network traffic between environments, but would require additional configuration and management of VNets and peering relationships.
Ultimately, the best solution will depend on the specific needs and constraints of your organization. It may be worth discussing with the Databricks solutions architect to determine the best approach for your use case.