karthik_p
Databricks Partner

@ashdam dis you tried DAB in Databricks, it has version control mechanism for jobs to , try to leverage that functionality 

karthik.p