- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-23-2024 03:25 PM
I have a similar use case. We have two different host for databricks, EU and NA. In some case we need to deploy a similar job in both hosts. To fix that, here how I did:
- Into job folder I created different job files, each one for one host. In aditional I created an empty job file, named job.yml
- src/
- jobs/
|--- job_EU.yml (job configuration for EU dbx)
|--- job_NA.yml (job configuration for NA dbx)
|--- job.yml (empty file)
- In databricks.yml I include only the empty file:
include:
- job.yml- Finally, at workflow file, in each job for databricks commands I edit the empty file pasting the content of the specific job file, depending on the host I want to deploy and run. This step must be done in each job that calls a databricks bundle command. If they are at the same job, just needed once:
deploy:
.
.
.
steps:
.
.
.
- run: cat jobs/job_EU.yml > jobs/job.yml
- run: databricks bundle deploy