topic Re: Permissions on job with SQL task in Data Engineering

Permissions on job with SQL task

stravi001 — Wed, 08 Jul 2026 18:52:15 GMT

Hello,

I am authenticated as GCP service account. I have to create query, run job as service principle and get result.

I'm both cases, I set permissions CAN_MANAGE for group "users" for both query and job. When I create the query with RunAsMode.OWNER, it gets limited permissions of invoker SP, but I can then access output of the task. When I create query with RunAsMode.VIEWER it gets required permissions of RunAs SP, but I am not able to access output of the task. I get error: insufficient privileges to access object queryExecutionSessions/*. As I understand from the roles description, it should have permissions with CAN_MANAGE.

Thank you for help.

Re: Permissions on job with SQL task

stravi001 — Thu, 09 Jul 2026 08:12:00 GMT

To make my question simpler. I have created Query with my personal account. I have set run as mode to viewer. I run it in task under SP, it succeeds, but I (as query owner) am not able to see output of that task, telling me I have no permissions for the query.

Re: Permissions on job with SQL task

balajij8 — Thu, 09 Jul 2026 12:47:09 GMT

What you are encountering is the expected behavior due to how Databricks isolates execution sessions and handles credential delegation between Jobs and SQL objects.

When you set a query to RunAsMode.VIEWER (Run as viewer), the query executes using the credentials of the Job's "Run as" service principal. Consequently, the resulting execution session (queryExecutionSessions) is owned entirely by that executing service principal. The CAN_MANAGE permission on the query object itself only grants you the ability to modify the query definition, it does not implicitly grant you access to another principal's execution session. Hence, you cannot view the output result.

With RunAsMode.OWNER (Run as owner), the query executes using your credentials, which grants you access to your own session outputs and limits the permissions to your personal access level rather than the service principal's.

You can follow below

If you want to simplify the permissions model and avoid cross-principal execution session errors entirely -You can unify the execution identity across the full cycle. Set the Service Principal as the Query Owner, configure the Query with RunAsMode.OWNER, and set the Job's "Run as" identity to that Service Principal. You can then manage data access directly via Unity Catalog. This ensures the Service Principal is always accessing its own execution sessions, and you act solely as the API caller triggering the job.

SQL File Task - You can switch the Job task type from a SQL Query task to a SQL File task. By storing the SQL statements directly in a file within the Workspace, you bypass the RunAsMode complexity completely. The SQL File task will strictly use the Job's "Run as" identity, streamlining the permission model.

More details here

Re: Permissions on job with SQL task

stravi001 — Thu, 09 Jul 2026 18:57:06 GMT

Hello @balajij8,

thank you for your answer.

Today I did testing with setup:

Job Creator: GCP SA
Job RunAs: SP
Query Owner: Personal account
Permissions: CAN_MANAGE for all users for both Query and Job

Both personal account and GCP SA were able to get the job output with RunAsMode.OWNER. Both got permissions error with RunAsMode.VIEWER.

Based on https://docs.databricks.com/gcp/en/security/auth/access-control/#job-acls, I would assume both should have permissions to view results.

My setup only allows me to authenticate as GCP SA, create Job with RunAs SP and create Query. That is pretty much all. And best would be to create the Query on the fly. Changing owner requires admin permissions.

Re: Permissions on job with SQL task

balajij8 — Fri, 10 Jul 2026 03:36:15 GMT

Hi @stravi001

What you are encountering on the SQL Query task is the expected behavior. You can use SQL File task as its behavior is more closely aligned with standard Job ACLs and completely skips the RunAsMode permission complexity.

SQL File Task

You can switch the Job task type from a SQL Query task to a SQL File task.By storing the SQL statements directly in a file you bypass the RunAsMode complexity. The SQL File task strictly uses the Job's Run as identity, which securely streamlines your permission model. More details here.

SQL Query Task

You can ask the Account Admin need to grant the Service Principal User role as below and validate once

To - The GCP SA and/or personal accounts (if both need to access the results).
On (Target) - The RunAs SP (the service principal configured in the job's Run as field).

Re: Permissions on job with SQL task

stravi001 — Fri, 10 Jul 2026 10:52:46 GMT

Hello @balajij8,

I tried to use SQL File Task and was able to achieve the result. Only issue was, that I was not able to get output using WorkspaceClient.jobs.get_run_output(). It worked will direct call of /api/2.2/jobs/runs/get-output endpoint.

Thank you for help