- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-21-2025 10:32 PM
Hi there @Venu,
You have to specify the names of those variables in databricks.yml as well
variables:
task_key:
metadata_schema:then you can reference them later in your jobs definition : ${var.task_key} or ${var.job_cluster_key} the way you did.
and have your variables file in .databricks/bundle/dev/variable-overrides.json
{
"task_key" : "abc",
"job_cluster_key" : "efg"
}This will make the code and run and deploy, but it wont solve your original problem of keeping the code clean.
For that you should divide your mappings into multiple bundle_configuration_files
like bundle.variables.yml and you can specifiy all your variables in that
similarly you can have it for bundle.resources.yml or bundle.targets.yml you can even have seperate files for each of your resources like pipelines, jobs, clusters etc.
keep them in a single folder
and then in your databricks.yml at the top
include:
- bundle.variables.yml
- bundle.resources.ymlYou can find more information about these in the databricks bundle configuration syntax docs : https://docs.databricks.com/aws/en/dev-tools/bundles/settings