Passing Different Job Parameters or Environment Variables Based on Targets in DABs

[azam-io]

Hi, I am currently using DABs in my project and have encountered an issue. Specifically, I need to pass different job parameters or environment variables depending on the target and for specific jobs.
Could you please provide guidance on how to approach this, or any best practices for handling this scenario?
Thank You

[Nivethan_Venkat]

Hi @azam-io,

Please find the best practices here for deploying jobs using Asset Bundles configuration: https://docs.databricks.com/aws/en/dev-tools/bundles/settings

If you would like to pass different parameters for your same job in different environments, you can segregate the jobs with respective configurations in respective target folders as below:

dev   ## folder containing respective jobs to be deployed in DEV
acc   ## folder containing respective jobs to be deployed in ACC
prd   ## folder containing respective jobs to be deployed in PRD
bundles.target.yml ## Here you are maintaining configurations for different environment

By having this approach you can pass different parameters to same job in different environments. In your bundles target, hopefully you will be pointing the respective environment variables for each envs.

Please let us know if you find this helpful or additional clarifications required.


Best Regards,
Nivethan V

[azam-io]

Hi @Nivethan_Venkat, thanks for your response, it's quite helpful. However i would like to know if there is any way to define those variables inside the resource file of the job instead of defining them in separate folders. You know like we do inside targets tag.

Regards,
Azam Ali