cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks bundle custom variables feature does not work for workspace host

AlbertWang
Contributor

Hi all,

I am not sure whether this is a bug. I followed the document.

My databricks.yml file:

 

bundle:
  name: test

variables:
  DATABRICKS_HOST:
    description: The host of the Databricks workspace.
  DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID:
    description: The Databricks workspace service principal for running jobs.

targets:
  dev:
    mode: development
    default: true
    workspace:
      host: ${var.DATABRICKS_HOST}
    run_as:
      service_principal_name: ${var.DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID}

 

Then I run the following command:

 

databricks bundle validate --var="DATABRICKS_HOST=https://aaaaa" --var="DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID=asdasdasdaa"

 

Then I get the following error:

 

Name: test
Target: dev
Workspace:
  Host: ${var.DATABRICKS_HOST}

 

Looks like the `${var.DATABRICKS_HOST}` custom variable substitution does not work for `workspace.host`.

If I modify my databricks.yml file to use the real value instead variable for `workspace.host`:

bundle:
  name: test

variables:
  DATABRICKS_HOST:
    description: The host of the Databricks workspace.
  DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID:
    description: The Databricks workspace service principal for running jobs.

targets:
  dev:
    mode: development
    default: true
    workspace:
      host: https://aaaaa
    run_as:
      service_principal_name: ${var.DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_ID}

Everything works including `databricks bundle deploy`. That means the custom variable substitution works for other configurations, but not for `workspace.host`.

What do you think?

Thank you.

Regards,

Albert

1 ACCEPTED SOLUTION

Accepted Solutions

filipniziol
New Contributor III

Hi @AlbertWang ,

The documentation states you cannot map workspace host:

filipniziol_0-1726644087377.png

 



View solution in original post

5 REPLIES 5

Witold
Contributor III

Hosts can't - and shouldn't be - variable. If you have multiple hosts, you need to define them properly in targets.

Thank you for your kind reply, @Witold. May I ask why host should not be variable? Is there any specific reason?

I use Azure DevOps pipelines as the CI/CD tool. So I define the host in variable groups. I don't want to define it again in a different location (like the Databricks bundle file).

Thanks again.

Regards,

Albert

Try to see it from the perspective of DAB, and not Azure DevOps. Your DevOps pipeline is only the orchestrating tool for DAB. Just think of a workspace being an environment, like a production environment. Do you want "anyone from the outside" (outside of DAB) decide what your production is? Is your production environment supposed to change every time you deploy?

Your answers will most likely be no for obvious reasons. And if you really want to change our target, your production environment to another workspace, it's usually associated with other, quite high efforts.

filipniziol
New Contributor III

Hi @AlbertWang ,

The documentation states you cannot map workspace host:

filipniziol_0-1726644087377.png

 



Thank you, @filipniziol. Even though I don't know why, but since it is a documented feature, it is not a bug ๐Ÿ™‚

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group