09-19-2022 12:23 PM
I am trying to write a process that will programmatically update the “run_as_user_name” parameter for all jobs in an Azure Databricks workspace, using powershell to interact with the Jobs API.
I have been trying to do this with a test job without success. I am, however, able to update different job params, such as job name, for the same job using powershell. This leads me to believe that its not possible to update run_as_user_name via the Jobs api, although I can't find this clearly stated anywhere in documentation.
I’ve tried with both the 2.0 and 2.1 versions of the API. The below powershell works successfully for updating the job name:
#set BaseUrl and headers
$BaseUrl = https://adb-***.azuredatabricks.net
$job_URL = "$BaseUrl/api/2.0/jobs/update"
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "Bearer XXXX")
#create the json for update
$JobId = '12345'
$paramDetail = '"new_settings": { "name": "change_to_this"}}'
$jobIdDetail = '"job_id":{0}' -f $JobId
$bodyContent = '{' + '{0},{1}' -f $jobIdDetail, $paramDetail + '}'
$bodyContent
#run the update
Invoke-RestMethod $job_URL -Method POST -Headers $headers -Body $bodyContent
However, if I change the following line:
I get no errors (in fact, I get no output at all), but nothing updates in the job within Databricks. Any thoughts on where I'm going wrong?
09-21-2022 03:53 AM
Answering my own question - I was using the wrong tool for the job. Using the "Permissions" API rather than the "Jobs" API is what was needed. The below powershell code does what i need:
#get existing owner
$current_perms_URL = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_details = Invoke-RestMethod $current_perms_URL -Method GET -Headers $headers
$current_owner = ($perms_details.access_control_list | where { $_.all_permissions.permission_level -eq "IS_OWNER" }).user_name
Write-Output "Current Owner is $current_owner"
#downgrade current owner perms to CAN_MANAGE and add/update desired owner. All other users left untouched
if ($current_owner -ne $desired_owner){
Write-Output "Updating owner to $desired_owner for JobId $JobId"
$bodyContent = '{ "access_control_list": [ { "user_name": ' + """$current_owner""" + ', "permission_level": "CAN_MANAGE" }, { "user_name": ' + """$desired_owner""" + ', "permission_level": "IS_OWNER" } ] }'
$bodyContent
$perms_url = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_url
Invoke-RestMethod $perms_url -Method PATCH -Headers $headers -Body $bodyContent
} else {
Write-Output "Owner already set to $desired_owner for JobId $JobId. No action taken"
}
09-20-2022 01:45 PM
Hi, is this a custom parameter?
09-20-2022 01:50 PM
No, its the setting which governs which user account ultimately runs a job. Its the equivalent of changing which user has "Is Owner" permissions on the job through the UI.
09-21-2022 03:53 AM
Answering my own question - I was using the wrong tool for the job. Using the "Permissions" API rather than the "Jobs" API is what was needed. The below powershell code does what i need:
#get existing owner
$current_perms_URL = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_details = Invoke-RestMethod $current_perms_URL -Method GET -Headers $headers
$current_owner = ($perms_details.access_control_list | where { $_.all_permissions.permission_level -eq "IS_OWNER" }).user_name
Write-Output "Current Owner is $current_owner"
#downgrade current owner perms to CAN_MANAGE and add/update desired owner. All other users left untouched
if ($current_owner -ne $desired_owner){
Write-Output "Updating owner to $desired_owner for JobId $JobId"
$bodyContent = '{ "access_control_list": [ { "user_name": ' + """$current_owner""" + ', "permission_level": "CAN_MANAGE" }, { "user_name": ' + """$desired_owner""" + ', "permission_level": "IS_OWNER" } ] }'
$bodyContent
$perms_url = "$BaseUrl/api/2.0/permissions/jobs/$JobId"
$perms_url
Invoke-RestMethod $perms_url -Method PATCH -Headers $headers -Body $bodyContent
} else {
Write-Output "Owner already set to $desired_owner for JobId $JobId. No action taken"
}
10-04-2023 07:34 AM
Solution you've submitted is a solution for different topic (permission to run job, the job still runs as the user in run_as_user_name field). Here is an example of changing "run_as_user_name"
Docs:
https://docs.databricks.com/api/azure/workspace/jobs/update (expand new_settings options)
import requests
import json
import logging
logging.basicConfig(
format="[%(asctime)s %(levelname)s %(filename)s:%(lineno)s - %(funcName)s()] %(message)s",
level=logging.INFO,
)
token = "dapi******-2"
new_settings = {
"job_id": 123,
"new_settings": {
"run_as": {"user_name": "name@company.com"}
}
}
data = json.dumps(new_settings)
logging.info("data=%s", data)
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
}
logging.debug("headers=%s", headers)
resp = requests.post(
url=f"https://<URL>/api/2.0/jobs/update",
data=data, headers=headers
)
logging.info("resp=%s %s", resp, resp.text)
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group