Hello!

To create a new job in Databricks Asset Bundles that refers to another job-task without hard-coding the job ID, use job parameters to pass dynamic values. Define a job parameter to hold the job ID once it’s created and use placeholders in your job configuration. This way, you can dynamically set the job ID when deploying the bundle.

Thanks for your response. However, it makes the process manual as when the first job is deployed, then get the first job-id and pass it to the second job to as a paramter-value and deploy the second job. I was looking for a way where the first job key (say name) can be used a reference parameter in the second job definition, so it stays dynamic.

Good question. If the job is created outside of the asset bundle, and exists before this bundle will be used for the first time you can use a lookup variable

variables:
  your_job_id:
    lookup:
      job: "your_job_name"

If it's created inside the same bundle you can reference it directly

${resources.jobs.<job-name>.id}

I've not tried these in anger though, so let me know how you get on.

https://docs.databricks.com/en/dev-tools/bundles/variables.html

Hi Team,

Yes this solution of referencing the variable ${resources.jobs.<job-name>.id} works within the same bundle and this should be a solution. I had tried that by creating multiple workflows and referencing that.

The lookup works. Here is how it can be used for a job existing outside the asset bundle

variables:
    my_jobid:
        description: Enter the Databricks Job name you want to refer.
        lookup:
          job: 'My Job1'

tasks:
        - task_key: Run MyJob1
          run_job_task:
            job_id: ${var.my_jobid}