Obtain access of Azure metastore storage account to configure Lifecycle management

[heathwinning]

I recently set up an Azure Databricks workspace with an automatically created metastore and metastore-level root storage within the metastore blob storage account. All the catalogs, schemas, and tables/volumes have been created without a specified or external location, so the data all reside in the metastore blob storage account under the container named "unity-catalog-storage".

Because of the "System deny assignment created by Azure Databricks" I have no direct access to the metastore blob storage account, and therefore cannot set the access tier of some large raw files to Cool, nor can I create lifecycle management policies to do this automatically.

I regret not setting up a separate storage account for catalogs, but if possible I'd love to avoid risking migration of lots of data in lots of tables. Is there a way to achieve the access required to configure Lifecycle management?

[Kaniz_Fatma]

Hi @heathwinning, To access your metastore blob storage account despite the "System deny assignment" issue, use Azure Managed Identities for secure authentication. Set up an access connector in the Azure portal for Azure Databricks, assign the `Storage Blob Data Contributor` role to the managed identity, and configure Unity Catalog to use this identity. This approach avoids the need for direct access and simplifies data management by leveraging managed identities for secure and efficient storage access. For detailed steps, see [Azure Managed Identities in Unity Catalog](https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/azure-managed-ident...).

[heathwinning]

Thanks for your response, @Kaniz_Fatma. I already have Unity Catalog configured using an access connector and managed identity, these were automatically created by the Databricks workspace initialisation. The issue I'm facing is that [Azure Blob Lifecycle Management Policies](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-policy-configure?tabs=azu...) require access to the storage container, but I am prevented from doing this by the Deny assignment.

I have tried

• logging in as the managed identity that has access to the storage container, but the managed identity is also blocked by a Deny assignment.
• creating another managed identity with access to the storage container, but the resource group is blocked by a Deny assignment

I want to know if there is a way around these Deny assignments as an administrator.