topic Re: How to preserve job run history when deploying with DABs in Data Engineering

How to preserve job run history when deploying with DABs

mkEngineer — Mon, 25 Aug 2025 11:14:54 GMT

I’m having an issue when deploying jobs with DABs. Each time I deploy changes, the existing job gets overwritten, the job name stays the same, but a new job ID is created. This causes the history of past runs to be lost.

Ideally, I’d like to update an existing job while keeping both the job name and the full run history. Has anyone found a best practice or configuration for handling this? Otherwise, I dont see the point of using DABs if history is lost for each?

Thanks in advance!

Re: How to preserve job run history when deploying with DABs

mkEngineer — Mon, 25 Aug 2025 11:15:10 GMT

@Walter_C @Alberto_Umana

Re: How to preserve job run history when deploying with DABs

Coffee77 — Mon, 25 Aug 2025 12:14:50 GMT

You are not deploying DAB correctly. You MUST use same ID for your jobs. Assigning only same names will create new jobs. If needed, ping me back for further details. It is a very typical issue when starting with DAB 🙂

Re: How to preserve job run history when deploying with DABs

Coffee77 — Mon, 25 Aug 2025 13:12:12 GMT

Yu must preserve content in green, that is a unique identifier for the Job basically. Property "name" acts as a kind of "alias" this case:

Re: How to preserve job run history when deploying with DABs

Coffee77 — Mon, 25 Aug 2025 13:54:07 GMT

By the way, this is a typical exam question in Databricks certification exams 😎

Re: How to preserve job run history when deploying with DABs

-werners- — Tue, 26 Aug 2025 07:49:31 GMT

do you run bundle destroy perhaps? because that will generate a new job id.

Re: How to preserve job run history when deploying with DABs

Coffee77 — Tue, 26 Aug 2025 10:13:03 GMT

Despite using different keys but same names, original jobs should remain indeed unless destroying them