- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-07-2026 02:31 AM
Hi @ChristianRRL,
No. As of now, Lakeflow Jobs doesn’t provide global, mutable variables that you can set from any task and read from any other task, regardless of scope. This is a current limitation of the platform...
I think you’ve already explored the supported patterns (job parameters, task values, etc.). I'm assuming you have a reason to keep the computation inside a separate child job. If so, the most robust option is to persist output_path to an external store (for example, a Delta table or a Unity Catalog volume / external location) in the child job. In the parent job, add a notebook task that reads that value and re-exposes it via dbutils.jobs.taskValues.set, and then reference it in downstream tasks using a dynamic value reference like {{tasks.<task_name>.values.output_path}}.
Using GET /api/2.1/jobs/runs/get-output doesn’t give you a global variable either. It’s read-only in the sense you can’t set a variable in Lakeflow Jobs with it. It works best for an external orchestrator pattern (external code runs Parent 1, calls get-output, then starts Parent 2 with that value as a job parameter).
Avoid using workspace DBFS for this kind of cross-job state. Prefer Unity Catalog managed storage instead.
If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***