Programmatically updating the “run_as_user_name” parameter for jobs

[peterwishart]

I am trying to write a process that will programmatically update the “run_as_user_name” parameter for all jobs in an Azure Databricks workspace, using powershell to interact with the Jobs API. 

I have been trying to do this with a test job without success. I am, however, able to update different job params, such as job name, for the same job using powershell. This leads me to believe that its not possible to update run_as_user_name via the Jobs api, although I can't find this clearly stated anywhere in documentation. 

I’ve tried with both the 2.0 and 2.1 versions of the API. The below powershell works successfully for updating the job name:

#set BaseUrl and headers

$BaseUrl = https://adb-***.azuredatabricks.net

$job_URL = "$BaseUrl/api/2.0/jobs/update"

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"

$headers.Add("Authorization", "Bearer XXXX")

#create the json for update

$JobId = '12345'

$paramDetail = '"new_settings": { "name": "change_to_this"}}'

$jobIdDetail = '"job_id":{0}' -f $JobId

$bodyContent = '{' + '{0},{1}' -f $jobIdDetail, $paramDetail + '}'

$bodyContent

#run the update

Invoke-RestMethod $job_URL -Method POST -Headers $headers -Body $bodyContent

However, if I change the following line:

• From: $paramDetail = '"new_settings": { "name": "change_to_this"}}'
• To: $paramDetail = '"new_settings": { "run_as_user_name": “my.user@company.com”}}'

I get no errors (in fact, I get no output at all), but nothing updates in the job within Databricks. Any thoughts on where I'm going wrong?

[Debayan]

Hi, is this a custom parameter?

[peterwishart]

No, its the setting which governs which user account ultimately runs a job. Its the equivalent of changing which user has "Is Owner" permissions on the job through the UI.

[peterwishart · Accepted solution]

Answering my own question - I was using the wrong tool for the job. Using the "Permissions" API rather than the "Jobs" API is what was needed. The below powershell code does what i need:

#get existing owner 
$current_perms_URL = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_details = Invoke-RestMethod $current_perms_URL -Method GET -Headers $headers
$current_owner = ($perms_details.access_control_list | where { $_.all_permissions.permission_level -eq "IS_OWNER" }).user_name
Write-Output "Current Owner is $current_owner"
 
#downgrade current owner perms to CAN_MANAGE and add/update desired owner. All other users left untouched
if ($current_owner -ne $desired_owner){
  Write-Output "Updating owner to $desired_owner for JobId $JobId"
  $bodyContent = '{ "access_control_list": [ { "user_name": ' + """$current_owner""" + ', "permission_level": "CAN_MANAGE" }, { "user_name": ' + """$desired_owner""" + ', "permission_level": "IS_OWNER" } ] }'
  $bodyContent
 
  $perms_url = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
  $perms_url
 
  Invoke-RestMethod $perms_url -Method PATCH -Headers $headers -Body $bodyContent
} else {
  Write-Output "Owner already set to $desired_owner for JobId $JobId. No action taken"
}

[baubleglue]

Solution you've submitted is a solution for different topic (permission to run job, the job still runs as the user in run_as_user_name field). Here is an example of changing "run_as_user_name"

Docs:
https://docs.databricks.com/api/azure/workspace/jobs/update (expand new_settings options)

import requests
import json
import logging

logging.basicConfig(
    format="[%(asctime)s %(levelname)s %(filename)s:%(lineno)s - %(funcName)s()] %(message)s",
    level=logging.INFO,
)


token = "dapi******-2"

new_settings = {
    "job_id": 123,
    "new_settings": {
        "run_as": {"user_name": "name@company.com"}
    }
}

data = json.dumps(new_settings)

logging.info("data=%s", data)

headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json",
}

logging.debug("headers=%s", headers)

resp = requests.post(
    url=f"https://<URL>/api/2.0/jobs/update",
    data=data, headers=headers
)

logging.info("resp=%s %s", resp, resp.text)