cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to preserve job run history when deploying with DABs

mkEngineer
New Contributor III

Hi

Iโ€™m having an issue when deploying jobs with DABs. Each time I deploy changes, the existing job gets overwritten,  the job name stays the same, but a new job ID is created. This causes the history of past runs to be lost.

Ideally, Iโ€™d like to update an existing job while keeping both the job name and the full run history. Has anyone found a best practice or configuration for handling this? Otherwise, I dont see the point of using DABs if history is lost for each? 

Thanks in advance!

6 REPLIES 6

mkEngineer
New Contributor III

Coffee77
Contributor III

You are not deploying DAB correctly. You MUST use same ID for your jobs. Assigning only same names will create new jobs. If needed, ping me back for further details. It is a very typical issue when starting with DAB ๐Ÿ™‚


Lifelong Learner Cloud & Data Solution Architect | https://www.youtube.com/@CafeConData

Coffee77
Contributor III

Yu must preserve content in green, that is a unique identifier for the Job basically. Property "name" acts as a kind of "alias" this case:

CAFE77_0-1756127417060.png

 


Lifelong Learner Cloud & Data Solution Architect | https://www.youtube.com/@CafeConData

Coffee77
Contributor III

By the way, this is a typical exam question in Databricks certification exams ๐Ÿ˜Ž


Lifelong Learner Cloud & Data Solution Architect | https://www.youtube.com/@CafeConData

-werners-
Esteemed Contributor III

do you run bundle destroy perhaps?  because that will generate a new job id.

Coffee77
Contributor III

Despite using different keys but same names, original jobs should remain indeed unless destroying them


Lifelong Learner Cloud & Data Solution Architect | https://www.youtube.com/@CafeConData

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now