cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databrikcs SDK - create new job using JSON

pavel_merkle
New Contributor II

Hello,

I am trying to create a Job via Databricks SDK. As input, I use the JSON generated via Workflows UI (Worklflows->Jobs->View YAML/JSON->JSON API->Create) generating pavel_job.json. When trying to run SDK function jobs.create as

dbk = WorkspaceClient(host=args.host, token=args.token, auth_type="pat")

job_dict = json.load(open("pavel_job.json"))
job = dbk.jobs.create(**job_dict)

This running into an error:

Error creating job 'pavel_job': 'dict' object has no attribute 'as_dict'

Can you please advice on how to use the generated JSON with SDK to create a Job?

3 REPLIES 3

mhiltner
New Contributor III
New Contributor III

Hey there! I have been using Volumes to get the files. It looks like this:

dbk = WorkspaceClient(host=args.host, token=args.token, auth_type="pat")

file_path = "/Volumes/{{your_catalog}}/{{your_schema}}/json_volumes/sample1.json"
content = dbutils.fs.head(file_path)
job_dict = eval(content)
job = dbk.jobs.create(**job_dict)
 
Your problem is probably when creating the dict, not calling the API function.

Hi mhiltner,

does this job creation work for you? Can you please share sample1.json? Do you run it in notebook? I am trying to run it via VSCode and other API calls work for me so I guess the json/dict I pass is the problem, but Ialready tried the simplest one.

What do you mean by not calling the API function when creating the dict?

mhiltner
New Contributor III
New Contributor III

it does work for me, using directly on a notebook cell (havent tried in vscode). 

This is my sample Json 

{
  "name": "Agregacao_Gold",
  "email_notifications": {},
  "webhook_notifications": {},
  "timeout_seconds": 0,
  "max_concurrent_runs": 1,
  "queue": {
    "enabled": True
  },
  "run_as": {
    "user_name": "mhiltner@community.com"
  }
}

 What i meant is that maybe your job_dict variable is not storing your job as a dict type and this could be causing an error. If you dont mind, feel free to share the result of printing your job_dict 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!