Impact of Updating DAB Root Path on Databricks Job Run History

[YS1]

Hello,

I’m using Databricks Asset Bundles (DABs) to orchestrate several workflows, and I’d like to update the root path where the bundle is stored.

Before making this change, I want to understand its impact:

1. Will changing the bundle root path remove or reset the existing workflow/job run history?

2. If so, is there a way to preserve the run history while changing the bundle location?

3. How does Databricks internally associate deployed jobs with their run history?

4. Where is this metadata stored, and is there documentation/articles that explains what happens in the backend?

If anyone has experience with this or can point me to an official source, I’d really appreciate it.

Thanks.

[WiliamRosa · Accepted solution]

Hi @YS1 
I did some research on this topic and found a few official documents and insights that may help:

Changing the bundle root path itself won’t remove or reset your job run history. Run history in Databricks is tied to the job_id that exists inside the workspace, not to where the Databricks Asset Bundle lives in your repo.

A DAB is just a local definition of assets. When you run databricks bundle deploy, the CLI updates (or creates) remote resources based on the job definition, but the run history stays attached to the same job as long as you’re still updating that existing job. The location of the bundle on disk isn’t part of that relationship.
Official docs on bundles:
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/bundles/

If the bundle config changes in a way that causes the CLI to create a new job instead of updating the old one, then yes — the new job starts with empty run history, simply because it’s a new job_id. The old job and its run history remain in the workspace unless deleted.
Jobs API reference (showing the run/job association):
https://docs.databricks.com/api/workspace/jobs/listruns

Databricks also documents run history behavior and retention here:
https://learn.microsoft.com/en-us/azure/databricks/jobs/monitor

Internally, the system associates run history with the job through job_id → run_id. For deeper analysis, you can also look into the system tables (“system.lakeflow.job_runs” and related tables) if they’re enabled in your workspace:
https://learn.microsoft.com/en-us/azure/databricks/jobs/monitor#system-tables

So the key point is:
If you keep deploying to the same job (same job_id), you keep the run history. Changing the bundle root won’t affect that. If you unintentionally create a new job, then the new one won’t have historical runs — but the old one stays intact.

[YS1]

@WiliamRosa Appreciate the detailed explanation — this clears it up!

[Coffee77]

Whenever you keep "job key name" in DAB, root folder should not have any impact. History is associated to the job irrespective of being deployed via DAB or updated manually. KR.

[YS1]

@Coffee77 Got it — thank you for confirming!