3 weeks ago
Hi all,
I am not sure whether this is a bug. I followed the document.
My databricks.yml file:
bundle:
name: test
variables:
DATABRICKS_HOST:
description: The host of the Databricks workspace.
DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID:
description: The Databricks workspace service principal for running jobs.
targets:
dev:
mode: development
default: true
workspace:
host: ${var.DATABRICKS_HOST}
run_as:
service_principal_name: ${var.DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID}
Then I run the following command:
databricks bundle validate --var="DATABRICKS_HOST=https://aaaaa" --var="DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID=asdasdasdaa"
Then I get the following error:
Name: test
Target: dev
Workspace:
Host: ${var.DATABRICKS_HOST}
Looks like the `${var.DATABRICKS_HOST}` custom variable substitution does not work for `workspace.host`.
If I modify my databricks.yml file to use the real value instead variable for `workspace.host`:
bundle:
name: test
variables:
DATABRICKS_HOST:
description: The host of the Databricks workspace.
DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID:
description: The Databricks workspace service principal for running jobs.
targets:
dev:
mode: development
default: true
workspace:
host: https://aaaaa
run_as:
service_principal_name: ${var.DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID}
Everything works including `databricks bundle deploy`. That means the custom variable substitution works for other configurations, but not for `workspace.host`.
What do you think?
Thank you.
Regards,
Albert
3 weeks ago
3 weeks ago
Hosts can't - and shouldn't be - variable. If you have multiple hosts, you need to define them properly in targets.
3 weeks ago
Thank you for your kind reply, @Witold. May I ask why host should not be variable? Is there any specific reason?
I use Azure DevOps pipelines as the CI/CD tool. So I define the host in variable groups. I don't want to define it again in a different location (like the Databricks bundle file).
Thanks again.
Regards,
Albert
3 weeks ago
Try to see it from the perspective of DAB, and not Azure DevOps. Your DevOps pipeline is only the orchestrating tool for DAB. Just think of a workspace being an environment, like a production environment. Do you want "anyone from the outside" (outside of DAB) decide what your production is? Is your production environment supposed to change every time you deploy?
Your answers will most likely be no for obvious reasons. And if you really want to change our target, your production environment to another workspace, it's usually associated with other, quite high efforts.
3 weeks ago
3 weeks ago
Thank you, @filipniziol. Even though I don't know why, but since it is a documented feature, it is not a bug ๐
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group