cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Programmatically updating the “run_as_user_name” parameter for jobs

peterwishart
New Contributor III

I am trying to write a process that will programmatically update the “run_as_user_name” parameter for all jobs in an Azure Databricks workspace, using powershell to interact with the Jobs API. 

I have been trying to do this with a test job without success. I am, however, able to update different job params, such as job name, for the same job using powershell. This leads me to believe that its not possible to update run_as_user_name via the Jobs api, although I can't find this clearly stated anywhere in documentation. 

I’ve tried with both the 2.0 and 2.1 versions of the API. The below powershell works successfully for updating the job name:

#set BaseUrl and headers

$BaseUrl = https://adb-***.azuredatabricks.net

$job_URL = "$BaseUrl/api/2.0/jobs/update"

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"

$headers.Add("Authorization", "Bearer XXXX")

#create the json for update

$JobId = '12345'

$paramDetail = '"new_settings": { "name": "change_to_this"}}'

$jobIdDetail = '"job_id":{0}' -f $JobId

$bodyContent = '{' + '{0},{1}' -f $jobIdDetail, $paramDetail + '}'

$bodyContent

#run the update

Invoke-RestMethod $job_URL -Method POST -Headers $headers -Body $bodyContent

However, if I change the following line:

  • From: $paramDetail = '"new_settings": { "name": "change_to_this"}}'
  • To: $paramDetail = '"new_settings": { "run_as_user_name": “my.user@company.com”}}'

I get no errors (in fact, I get no output at all), but nothing updates in the job within Databricks. Any thoughts on where I'm going wrong?

1 ACCEPTED SOLUTION

Accepted Solutions

peterwishart
New Contributor III

Answering my own question - I was using the wrong tool for the job. Using the "Permissions" API rather than the "Jobs" API is what was needed. The below powershell code does what i need:

#get existing owner 
$current_perms_URL = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_details = Invoke-RestMethod $current_perms_URL -Method GET -Headers $headers
$current_owner = ($perms_details.access_control_list | where { $_.all_permissions.permission_level -eq "IS_OWNER" }).user_name
Write-Output "Current Owner is $current_owner"
 
#downgrade current owner perms to CAN_MANAGE and add/update desired owner. All other users left untouched
if ($current_owner -ne $desired_owner){
  Write-Output "Updating owner to $desired_owner for JobId $JobId"
  $bodyContent = '{ "access_control_list": [ { "user_name": ' + """$current_owner""" + ', "permission_level": "CAN_MANAGE" }, { "user_name": ' + """$desired_owner""" + ', "permission_level": "IS_OWNER" } ] }'
  $bodyContent
 
  $perms_url = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
  $perms_url
 
  Invoke-RestMethod $perms_url -Method PATCH -Headers $headers -Body $bodyContent
} else {
  Write-Output "Owner already set to $desired_owner for JobId $JobId. No action taken"
}

View solution in original post

4 REPLIES 4

Debayan
Databricks Employee
Databricks Employee

Hi, is this a custom parameter?

peterwishart
New Contributor III

No, its the setting which governs which user account ultimately runs a job. Its the equivalent of changing which user has "Is Owner" permissions on the job through the UI.

peterwishart
New Contributor III

Answering my own question - I was using the wrong tool for the job. Using the "Permissions" API rather than the "Jobs" API is what was needed. The below powershell code does what i need:

#get existing owner 
$current_perms_URL = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_details = Invoke-RestMethod $current_perms_URL -Method GET -Headers $headers
$current_owner = ($perms_details.access_control_list | where { $_.all_permissions.permission_level -eq "IS_OWNER" }).user_name
Write-Output "Current Owner is $current_owner"
 
#downgrade current owner perms to CAN_MANAGE and add/update desired owner. All other users left untouched
if ($current_owner -ne $desired_owner){
  Write-Output "Updating owner to $desired_owner for JobId $JobId"
  $bodyContent = '{ "access_control_list": [ { "user_name": ' + """$current_owner""" + ', "permission_level": "CAN_MANAGE" }, { "user_name": ' + """$desired_owner""" + ', "permission_level": "IS_OWNER" } ] }'
  $bodyContent
 
  $perms_url = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
  $perms_url
 
  Invoke-RestMethod $perms_url -Method PATCH -Headers $headers -Body $bodyContent
} else {
  Write-Output "Owner already set to $desired_owner for JobId $JobId. No action taken"
}

baubleglue
New Contributor II

 

 

Solution you've submitted is a solution for different topic (permission to run job, the job still runs as the user in run_as_user_name field). Here is an example of changing "run_as_user_name"

Docs:
https://docs.databricks.com/api/azure/workspace/jobs/update (expand new_settings options)

 

import requests
import json
import logging

logging.basicConfig(
    format="[%(asctime)s %(levelname)s %(filename)s:%(lineno)s - %(funcName)s()] %(message)s",
    level=logging.INFO,
)


token = "dapi******-2"

new_settings = {
    "job_id": 123,
    "new_settings": {
        "run_as": {"user_name": "name@company.com"}
    }
}

data = json.dumps(new_settings)

logging.info("data=%s", data)

headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json",
}

logging.debug("headers=%s", headers)

resp = requests.post(
    url=f"https://<URL>/api/2.0/jobs/update",
    data=data, headers=headers
)

logging.info("resp=%s %s", resp, resp.text)

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group