Job cluster support in jobs/runs/submit API

[hanish]

We are using jobs/runs/submit API of databricks to create and trigger a one-time run with new_cluster and existing_cluster configuration.

We would like to check if there is provision to pass "job_clusters" in this API to reuse the same cluster across multiple tasks of job?

We are not able to find this in Databricks API Doc.

[daniel_sahal]

@Hanish Bansal​ 

Reading Jobs API updates, they've added support for shared job cluster.

https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/jobs-api-updates?source=recommenda...

To configure shared job clusters, include a JobCluster array in the JobSettings object. You can specify a maximum of 100 clusters per job. The following is an example of an API 2.1 response for a job configured with two shared clusters

[hanish]

I tried with a sample but got error "Shared job cluster feature is not supported in runs/submit API."

I found the same in documentation "https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/jobs-api-updates?source=recommendations#--runs-submit" that runs submit request does not support shared job clusters.

Not sure if there is any specific reason to not supporting shared job clusters in this API.

[Anonymous]

@Hanish Bansal​ Shared job cluster for  jobs/runs/submit API is not supported at the moment.

[giladba]

Hi, 
Any update on this issue? Is a shared job cluster supported for the jobs/runs/submit API or planned to be supported in the near future?

[Nagrjuna]

Hi, 

Any update on the above mentioned issue? Unable to submit a one time new job run (api/2.0 or 21/jobs/runs/submit) with shared job cluster or one new cluster has to be used for all TASKs in the job