ashraf1395
Honored Contributor

Hi there @Venu,

You have to specify the names of those variables in databricks.yml as well

variables:
 task_key:
 metadata_schema:

then you can reference them later in your jobs definition : ${var.task_key} or ${var.job_cluster_key} the way you did.

and have your variables file in .databricks/bundle/dev/variable-overrides.json

{
 "task_key" : "abc",
 "job_cluster_key" : "efg"
}

This will make the code and run and deploy, but it wont solve your original problem of keeping the code clean.

For that you should divide your mappings into multiple bundle_configuration_files

like bundle.variables.yml and you can specifiy all your variables in that

similarly you can have it for bundle.resources.yml or bundle.targets.yml  you can even have seperate files for each of your resources like pipelines, jobs, clusters etc.

keep them in a single folder 

and then in your databricks.yml at the top 

include:

- bundle.variables.yml

- bundle.resources.yml

You can find more information about these in the databricks bundle configuration syntax docs : https://docs.databricks.com/aws/en/dev-tools/bundles/settings