topic Re: How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job? in Data Engineering

How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job?

NancyX — Mon, 23 Jun 2025 13:33:40 GMT

Is it possible to pass dynamic parameters, such as job.run_id to a pipeline_task within a Databricks Workflow job?

Re: How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job?

dataminion01 — Mon, 23 Jun 2025 18:40:13 GMT

yes. it's in the Parameters section

Re: How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job?

NancyX — Tue, 24 Jun 2025 02:56:23 GMT

Does this option available for Task type -Pipeline? I am able to see this for example notebook task but not for pipeline task

Re: How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job?

Ashwin_DSA — Fri, 05 Jun 2026 15:51:37 GMT

Hi @NancyX,

Apologies for the delayed follow-up. Closing the loop here with the current state. Yes, pipeline tasks do support parameters, and you can use dynamic value references such as {{job.run_id}} when configuring a pipeline task’s Parameters field. The relevant docs are Pipeline task for jobs, Access parameter values from a task, and What is a dynamic value reference?. The docs also note that pipeline task parameters are in Beta, so if you do not see the Parameters section for a Pipeline task in the UI, that is usually because the feature is gated by preview availability in the workspace.

If you want to pass the workflow run ID to the pipeline, set a pipeline task parameter like run_id = {{job.run_id}}. Inside the pipeline, those values can then be referenced as pipeline parameters; for SQL pipelines, the documented approach is named parameters, while for Python pipeline code, the docs recommend continuing to use the pipeline configuration field for runtime values.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

Re: How to pass Dynamic parameters like job.run_id to a pipeline_task in Databricks workflow job?

mazeem-arbisoft — Fri, 05 Jun 2026 16:58:23 GMT

Hi @NancyX ,

In addition to @Ashwin_DSA 's explanation, if you need to do the same from Declarative Automation Bundles (DABs formerly) workflow YAML markup, here are a couple examples from one of my jobs.

Run ID:

- name: job_run_id default: "{{job.run_id}}"

Referring your own custom job params in task params for example, here is an example of spark task referring to the custom params defined under the workflow job:

tasks: - task_key: my_task spark_python_task: python_file: ../src/my_python_file.py parameters: - "--is-historical-run" - "{{job.parameters.is_historical_run}}"