cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Make API Call to run job

B_J_Innov
New Contributor III

Hi everyone,

I want to trigger a run for a job using API Call.

Here's my code

"""

import shlex
import subprocess

def call_curl(curl):
args = shlex.split(curl)
process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
return stdout, stderr

if __name__ == '__main__':
curl = '''curl --netrc --request POST \
https://<databricks-instance>/api/2.1/jobs/run-now \
--data {"job_id": 123456}
\
| jq .
'''
stdout, stderr = call_curl(curl)
print(stdout)
print(stderr)"""

This returns : """

b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>\n<title>Error 401 Unauthorized</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem accessing /api/2.0/jobs/run-now. Reason:\n<pre>    Unauthorized</pre></p>\n</body>\n</html>\n'
b'  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   266  100   258  100     8   1601     49 --:--:-- --:--:-- --:--:--  1612\ncurl: (3) [globbing] unmatched close brace/bracket at pos 16\ncurl: (6) Could not resolve host: |; Unknown error\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: jq; Unknown error\ncurl: (6) Could not resolve host: .; Unknown error\n'
Script duration: 0.8s.

"""

I understand that I need to put credentials of some sort somewhere but in the documentation I checked, it doesn't show where or how I should do it.

https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#run-now

Someone already did this ? I would appreciate if someone could help me.

 

Thanks

 

2 REPLIES 2

Kaniz
Community Manager
Community Manager

Hi @B_J_Innov, To resolve the "Unauthorized" error when triggering a Databricks job via the API, ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers. Verify that your API endpoint is correct (e.g., `/api/2.0/jobs/runs/submit`), and provide the correct job ID and settings in your request data. Modify your Python code to include the access token and correct endpoint, and adjust the `settings` parameter to match your job configuration. If you continue to encounter issues, double-check your authentication setup and ensure that your access token has the necessary permissions to trigger the job.

B_J_Innov
New Contributor III

Hi Kaniz, Thank you for your answer
"ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers."
Wher can I generate my PAT and where is the Bearer token in my request header ? 

Do you have an example of a request that returns a successful run ?

"/api/2.0/jobs/runs/submit" : I took this from the documentation I linked but unfortunately there was no complete example

"provide the correct job ID and settings in your request data." I changed the job ID when I linked my code with the databricks instance to be safe, but in my initial code it was correct.

I don't have any setting in my job configuration, it's all default, I just use the cluster from my databricks instance, it should work right ?


"double-check your authentication setup and ensure that your access token has the necessary permissions to trigger the job." Where can I check this please ? 

Thanks in advance for your answers and your time

 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!