Make API Call to run job

B_J_Innov — Tue, 27 Feb 2024 16:39:30 GMT

Hi everyone,

I want to trigger a run for a job using API Call.

Here's my code

"""

import shlex
import subprocess

def call_curl(curl):
args = shlex.split(curl)
process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
return stdout, stderr

if __name__ == '__main__':
curl = '''curl --netrc --request POST \
https://<databricks-instance>/api/2.1/jobs/run-now \
--data {"job_id": 123456}
\
| jq .
'''
stdout, stderr = call_curl(curl)
print(stdout)
print(stderr)"""

This returns : """

b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>\n<title>Error 401 Unauthorized</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem accessing /api/2.0/jobs/run-now. Reason:\n<pre> Unauthorized</pre></p>\n</body>\n</html>\n'
b' % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 266 100 258 100 8 1601 49 --:--:-- --:--:-- --:--:-- 1612\ncurl: (3) [globbing] unmatched close brace/bracket at pos 16\ncurl: (6) Could not resolve host: |; Unknown error\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: jq; Unknown error\ncurl: (6) Could not resolve host: .; Unknown error\n'
Script duration: 0.8s.

"""

I understand that I need to put credentials of some sort somewhere but in the documentation I checked, it doesn't show where or how I should do it.

https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#run-now

Someone already did this ? I would appreciate if someone could help me.

Thanks

Re: Make API Call to run job

B_J_Innov — Thu, 29 Feb 2024 08:54:55 GMT

Hi Kaniz, Thank you for your answer
"ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers."
Wher can I generate my PAT and where is the Bearer token in my request header ?

Do you have an example of a request that returns a successful run ?

"/api/2.0/jobs/runs/submit" : I took this from the documentation I linked but unfortunately there was no complete example

"provide the correct job ID and settings in your request data." I changed the job ID when I linked my code with the databricks instance to be safe, but in my initial code it was correct.

I don't have any setting in my job configuration, it's all default, I just use the cluster from my databricks instance, it should work right ?

"double-check your authentication setup and ensure that your access token has the necessary permissions to trigger the job." Where can I check this please ?

Thanks in advance for your answers and your time

topic Make API Call to run job in Get Started Discussions

Make API Call to run job

Re: Make API Call to run job