Obtain access of Azure metastore storage account to configure Lifecycle management
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2024 05:13 PM
I recently set up an Azure Databricks workspace with an automatically created metastore and metastore-level root storage within the metastore blob storage account. All the catalogs, schemas, and tables/volumes have been created without a specified or external location, so the data all reside in the metastore blob storage account under the container named "unity-catalog-storage".
Because of the "System deny assignment created by Azure Databricks" I have no direct access to the metastore blob storage account, and therefore cannot set the access tier of some large raw files to Cool, nor can I create lifecycle management policies to do this automatically.
I regret not setting up a separate storage account for catalogs, but if possible I'd love to avoid risking migration of lots of data in lots of tables. Is there a way to achieve the access required to configure Lifecycle management?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-07-2024 02:38 PM
Thanks for your response, @Retired_mod. I already have Unity Catalog configured using an access connector and managed identity, these were automatically created by the Databricks workspace initialisation. The issue I'm facing is that [Azure Blob Lifecycle Management Policies](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-policy-configure?tabs=azu...) require access to the storage container, but I am prevented from doing this by the Deny assignment.
I have tried
- logging in as the managed identity that has access to the storage container, but the managed identity is also blocked by a Deny assignment.
- creating another managed identity with access to the storage container, but the resource group is blocked by a Deny assignment
I want to know if there is a way around these Deny assignments as an administrator.

