To connect Databricks to OneLake using Unity Catalog and access data with a service principal, and to address the "PERMISSION_DENIED" error you encountered, here are the key points and steps:

Use External Location with Unity Catalog

• When using Unity Catalog, you typically do not access cloud storage directly by URI. Instead, you create an external location in Unity Catalog that references your OneLake storage path.

• External locations allow controlled, managed access to storage with permissions enforced via Unity Catalog rather than raw storage permissions.

• Creating and managing external locations requires appropriate privileges in the metastore (e.g., metastore admin or external location owner role).​

Permissions for OneLake Storage Access

• The "PERMISSION_DENIED" error indicates that the service principal does not have sufficient permissions on the OneLake storage.

• You need to grant your Databricks service principal both Azure RBAC roles and OneLake workspace access:

  • At the Azure level, assign your service principal roles like Storage Blob Data Contributor or Storage Account Contributor on the OneLake storage account or relevant resource group.

  • Within OneLake (Microsoft Fabric workspace), assign the contributor role or equivalent access for the service principal to the target Fabric workspace or lakehouse.

• The access control model for OneLake uses deny-by-default, so explicit granting in both Azure portal (IAM role assignments) and Fabric workspace access control is required.​

Steps to Grant Permissions

1. In the Azure portal, go to your OneLake storage account or resource group.

2. Open "Access Control (IAM)" and add a role assignment for your service principal with the role Storage Blob Data Contributor or Storage Account Contributor.

3. In the Fabric portal, navigate to the target workspace, open Manage Access, and add your service principal with at least the Contributor role so it can access lakehouse data.

4. Confirm that the service principal has the necessary permissions to authenticate and read from the storage URI.

Using External Location in Databricks

• Create an external location in Databricks referencing your OneLake path using the same service principal/credential.

• Assign this external location to the appropriate workspace(s).

• Use Unity Catalog tables via this external location for fine-grained access control rather than mounting the storage manually.​

About Mounting OneLake Storage

• Mounting OneLake storage via DBFS is generally not recommended when using Unity Catalog.

• Instead, use external locations tied to Unity Catalog and the service principal access model for secured, governed data access.

• Mount attempts often fail with permission errors due to missing Contributor roles or managed identity rights on the OneLake storage account.​

This guidance should help resolve permission issues and align with best practices using Unity Catalog external locations for OneLake data in Databricks