Cloning of Workflow from One env to different env using Job API

[ameet9257]

Hi Team,

One of my team members recently shared one requirement: he wants to migrate the 10 Workflows from the sandbox to the dev environment to run his model in dev env.

I wanted to move all these workflows in an automated way and one of the solutions that came to my mind is Databricks Job API.

Below are the quick steps that I have followed.

1. Get the existing workflow configuration from the Databricks UI. 

Open existing workflow --> Click on View JSON 

2. Choose the Create option and copy the JSON.

3. Call the JOB API and pass the copy JSON to the API body.

POST https://XXXXXX.azuredatabricks.net/api/2.1/jobs/create

Using these three steps, I migrated the workflow to a new environment. 

I think the best way is to go through the terraform and CI-CD for production env but if there is no such thing in place I found this as the quickest option. 

Please do share your solutions as well. 

#Job API #databricks #workflows

[Stefan-Koch]

Hi Amet

Databricks Asset Bundles are designed precisely for the requirements you have.

https://docs.databricks.com/en/dev-tools/bundles/index.html

You can also transfer existing jobs that were created manually into a bundle. This works like this, for example: 

databricks bundle generate job --existing-job-id 6565621249

[ameet9257]

@Stefan-Koch 

Thanks. This looks interesting and I will try this.