cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to preserve job run history when deploying with DABs

mkEngineer
New Contributor III

Hi

Iโ€™m having an issue when deploying jobs with DABs. Each time I deploy changes, the existing job gets overwritten,  the job name stays the same, but a new job ID is created. This causes the history of past runs to be lost.

Ideally, Iโ€™d like to update an existing job while keeping both the job name and the full run history. Has anyone found a best practice or configuration for handling this? Otherwise, I dont see the point of using DABs if history is lost for each? 

Thanks in advance!

6 REPLIES 6

mkEngineer
New Contributor III

Coffee77
New Contributor II

You are not deploying DAB correctly. You MUST use same ID for your jobs. Assigning only same names will create new jobs. If needed, ping me back for further details. It is a very typical issue when starting with DAB ๐Ÿ™‚

https://www.youtube.com/@CafeConData

Coffee77
New Contributor II

Yu must preserve content in green, that is a unique identifier for the Job basically. Property "name" acts as a kind of "alias" this case:

CAFE77_0-1756127417060.png

 

https://www.youtube.com/@CafeConData

Coffee77
New Contributor II

By the way, this is a typical exam question in Databricks certification exams ๐Ÿ˜Ž

https://www.youtube.com/@CafeConData

-werners-
Esteemed Contributor III

do you run bundle destroy perhaps?  because that will generate a new job id.

Coffee77
New Contributor II

Despite using different keys but same names, original jobs should remain indeed unless destroying them

https://www.youtube.com/@CafeConData

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now