Using run_job_task in Databricks Asset Bundles

TimReddick
Contributor

Do Databrick Asset Bundles support run_job_task tasks?

I've made various attempts to add a run_job_task with a specified job_id. See my the code_snippet below. I tried substituting the job_id using ${...} syntax, as well as three other ways which I've commented out. I've tried to pass the id as both a string and an int.

When I pass a string, it returns: "error unmarshaling JSON: json: cannot unmarshal string into Go struct field RunJobTask.resources.jobs.tasks.run_job_task.job_id of type int."

When I pass an int, it returns "Error: cannot create job: Job 0 does not exist." However, the when I pass the int bundle validate does succeed.

 

resources:
  jobs:
    product_dimension_template:
      name: "Product Dimension Template"
      tasks:
        - task_key: product_dimension
          notebook_task:
            notebook_path: ./app/product_dim.py
          existing_cluster_id: 0723-185453-mypxim3x # tr_compute_autoscale
      tags:
        product: FCOE_measure_prototype

    product_dimension_ACDS_product_div:
      name: "Product Dimension - ACDS Product Div"
      tasks:
        - task_key: product_dimension_ACDS_product_div
          run_job_task:
            job_id: ${resources.jobs.product_dimension_template.id}
            # job_id: "${resources.jobs.product_dimension_template.id}"
            # job_id: "1032011550656546"
            # job_id: 1032011550656546
            notebook_params:
              product_dim_source": ACDS_product_div
              product_dim_source_path": ${var.ACDS_product_div_path}
              product_dim_checkpoint_path": ${var.ACDS_product_div_checkpoint_path}
              product_dim_table_path": ${var.product_dim_table_path}
      tags:
        product: FCOE_measure_prototype