filipniziol
Esteemed Contributor

Hi @smit_tw ,

yes, it is possible to do so, here is the sample yml:

resources:
  jobs:
    sample_job:
      name: sample_job
      tasks:
        - task_key: JobTask
          run_job_task:
            job_id: 1094194179990459
        - task_key: DltTask1
          depends_on:
            - task_key: JobTask
          pipeline_task:
            pipeline_id: da5fa00c-33b6-4850-8ea3-53f6e8d4b0e9
            full_refresh: false
        - task_key: DltTask2
          depends_on:
            - task_key: DltTask1
          pipeline_task:
            pipeline_id: da5fa00c-33b6-4850-8ea3-53f6e8d4b0e9
            full_refresh: false
      queue:
        enabled: true

The possible improvement is to use lookup to to reference jobs and pipelines by their name rather than their ids.

Check this discussion:
https://community.databricks.com/t5/data-engineering/getting-job-id-dynamically-to-create-another-jo...

 

View solution in original post