Make API Call to run job
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-27-2024 08:39 AM
Hi everyone,
I want to trigger a run for a job using API Call.
Here's my code
"""
import shlex
import subprocess
def call_curl(curl):
args = shlex.split(curl)
process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
return stdout, stderr
if __name__ == '__main__':
curl = '''curl --netrc --request POST \
https://<databricks-instance>/api/2.1/jobs/run-now \
--data {"job_id": 123456}
\
| jq .
'''
stdout, stderr = call_curl(curl)
print(stdout)
print(stderr)"""
This returns : """
b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>\n<title>Error 401 Unauthorized</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem accessing /api/2.0/jobs/run-now. Reason:\n<pre> Unauthorized</pre></p>\n</body>\n</html>\n'
b' % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 266 100 258 100 8 1601 49 --:--:-- --:--:-- --:--:-- 1612\ncurl: (3) [globbing] unmatched close brace/bracket at pos 16\ncurl: (6) Could not resolve host: |; Unknown error\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: jq; Unknown error\ncurl: (6) Could not resolve host: .; Unknown error\n'
Script duration: 0.8s.
"""
I understand that I need to put credentials of some sort somewhere but in the documentation I checked, it doesn't show where or how I should do it.
https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#run-now
Someone already did this ? I would appreciate if someone could help me.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-29-2024 12:54 AM
Hi Kaniz, Thank you for your answer
"ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers."
Wher can I generate my PAT and where is the Bearer token in my request header ?
Do you have an example of a request that returns a successful run ?
"/api/2.0/jobs/runs/submit" : I took this from the documentation I linked but unfortunately there was no complete example
"provide the correct job ID and settings in your request data." I changed the job ID when I linked my code with the databricks instance to be safe, but in my initial code it was correct.
I don't have any setting in my job configuration, it's all default, I just use the cluster from my databricks instance, it should work right ?
"double-check your authentication setup and ensure that your access token has the necessary permissions to trigger the job." Where can I check this please ?
Thanks in advance for your answers and your time

