System generated Service Principal -App

[rkhbo3003]

I created an App and creating agent to generate ETL pipeline through prompts. User interaction is with app, however when I do schema check on ADLS it says user does not have access. I cannot see SP in UI as it is system generated, even though I have provided all access to all workspace users, still issue exist.

[balajij8]

@rkhbo3003 

Databricks provisions a dedicated, system generated service principal to act as the App's identity. This service principal evaluates permissions completely independently of any workspace user and hence granting access to all workspace users does not cover the app. Databricks uses this identity to evaluate the app's permissions independently of any user so the app can only access resources explicitly granted to it. While the SP does not appear in the workspace admin users list, you can find its unique ID directly on the Authorization tab of app in the Databricks UI to use in the GRANT statements for providing access.

You can grant the app's service principal Unity Catalog privileges based on how the agent accesses the data to fix the ADLS access issue

• Managed/External Tables - If the agent queries existing external tables registered in Unity Catalog that point to ADLS, the SP requires USE CATALOG on the catalog, USE SCHEMA on the schema and SELECT on the table.
• Direct File Reads - If the agent reads files directly from ADLS, you can assign GRANT READ FILES ON EXTERNAL LOCATION <location_name> TO <app-sp-id> according to the Unity Catalog privileges reference.
  Volumes - If the data access path goes through a Unity Catalog volume, give GRANT READ VOLUME ON VOLUME <volume_name> TO `<app-sp-id>

GRANT READ FILES ON EXTERNAL LOCATION <location_name> TO <app-sp-id>
GRANT READ VOLUME ON VOLUME <volume_name> TO `<app-sp-id>

[rkhbo3003]

I tried that but getting access issues

[balajij8]

You can check the authorization mode the app is using. If the app is configured for user authorization (acting on behalf of the user) rather than app authorization (using the service principal’s identity), the effective permissions come from the interacting user, not the service principal. Error may indicate that the user interacting with the app doesn’t have access to ADLS, even though the service principal does in that case.

You can verify it in the app’s Authorization tab. If it’s using user authorization, the user will need the same Unity Catalog grants. Check if below grants are provided.

GRANT CAN USE ON WAREHOUSE <warehouse_name> TO `<app-sp-id>`;

GRANT USE CATALOG ON CATALOG <catalog_name> TO `<app-sp-id>`;
GRANT USE SCHEMA ON SCHEMA <catalog_name>.<schema_name> TO `<app-sp-id>`;

GRANT SELECT ON TABLE <catalog_name>.<schema_name>.<table_name> TO `<app-sp-id>`;

GRANT READ FILES ON EXTERNAL LOCATION <external_location_name> TO `<app-sp-id>`;

GRANT CREATE EXTERNAL TABLE ON EXTERNAL LOCATION <external_location_name> TO `<app-sp-id>`;
GRANT CREATE TABLE ON SCHEMA <catalog_name>.<schema_name> TO `<app-sp-id>`;

 Check if the correct SP id is used in all grants