cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Make API Call to run job

B_J_Innov
New Contributor III

Hi everyone,

I want to trigger a run for a job using API Call.

Here's my code

"""

import shlex
import subprocess

def call_curl(curl):
args = shlex.split(curl)
process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
return stdout, stderr

if __name__ == '__main__':
curl = '''curl --netrc --request POST \
https://<databricks-instance>/api/2.1/jobs/run-now \
--data {"job_id": 123456}
\
| jq .
'''
stdout, stderr = call_curl(curl)
print(stdout)
print(stderr)"""

This returns : """

b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>\n<title>Error 401 Unauthorized</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem accessing /api/2.0/jobs/run-now. Reason:\n<pre>    Unauthorized</pre></p>\n</body>\n</html>\n'
b'  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   266  100   258  100     8   1601     49 --:--:-- --:--:-- --:--:--  1612\ncurl: (3) [globbing] unmatched close brace/bracket at pos 16\ncurl: (6) Could not resolve host: |; Unknown error\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: jq; Unknown error\ncurl: (6) Could not resolve host: .; Unknown error\n'
Script duration: 0.8s.

"""

I understand that I need to put credentials of some sort somewhere but in the documentation I checked, it doesn't show where or how I should do it.

https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#run-now

Someone already did this ? I would appreciate if someone could help me.

 

Thanks

 

1 REPLY 1

Hi Kaniz, Thank you for your answer
"ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers."
Wher can I generate my PAT and where is the Bearer token in my request header ? 

Do you have an example of a request that returns a successful run ?

"/api/2.0/jobs/runs/submit" : I took this from the documentation I linked but unfortunately there was no complete example

"provide the correct job ID and settings in your request data." I changed the job ID when I linked my code with the databricks instance to be safe, but in my initial code it was correct.

I don't have any setting in my job configuration, it's all default, I just use the cluster from my databricks instance, it should work right ?


"double-check your authentication setup and ensure that your access token has the necessary permissions to trigger the job." Where can I check this please ? 

Thanks in advance for your answers and your time

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group