cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Asset Bundles: how to link a bundle-managed Genie space to an AI/BI dashboard across environments (o

koosha
New Contributor III

Hi all,

I'm managing an AI/BI dashboard and a Genie space with Databricks Asset Bundles (CLI v1.20, engine: direct), deployed to three workspaces (dev / test / prod) via Azure DevOps.

The problem When I link the Genie space to the dashboard in the UI, the exported .lvdash.json gets this block:

"uiSettings": {
  "genieSpace": {
    "isEnabled": true,
    "overrideId": "<genie_space_id>",
    "enablementMode": "ENABLED"
  }
}

The space ID is different in each workspace, because the bundle creates a new space per target. But:

  • the content of the .lvdash.json file is not interpolated, so I can't put ${resources.genie_spaces.my_space.id} there;
  • the dashboard resource has dataset_catalog / dataset_schema overrides, but nothing equivalent for the Genie space.

What I'd like to avoid

  • Manual steps after each deploy.

Current workaround (not satisfied with it) I put a placeholder ("overrideId": "__GENIE_SPACE_ID__") in the file. In the CD pipeline, before bundle deploy, a script:

  1. reads the space ID from databricks bundle summary -t <env> -o json;
  2. on the very first deploy to an environment, when the space doesn't exist yet, deploys once with an empty overrideId to create the space, then re-reads the ID;
  3. replaces the placeholder and runs bundle deploy.

It works on paper, but it needs a double deploy the first time and custom logic in the pipeline.

Thanks!

3 REPLIES 3

juanlozadab
Contributor

Hi! Iโ€™d try serialized_dashboard before keeping the double-deploy workaround.

Thereโ€™s some concrete support for that approach: the CLI v1.20.0 source explicitly allows serialized_dashboard to contain an inline YAML object. Also, Databricksโ€™ official app-with-Genie example references a bundle-managed space using:

${resources.genie_spaces.nyc_taxi_genie.space_id}

So there is an official example of referencing the space identifier. The part I havenโ€™t verified is using that reference inside the dashboardโ€™s uiSettings.genieSpace.overrideId with the direct engine.

Iโ€™d test moving the complete dashboard definition into serialized_dashboard, removing file_path, and using the corresponding space reference for overrideId. Both resources would be in the same bundle for this test.

In our project, we generate target-specific Genie JSON from templates before deployment. That handles values already known, such as the catalog, but we donโ€™t currently automate this dashboard-to-Genie link. I wouldnโ€™t present our implementation as proof that your first-deployment issue is solved.

The deciding test would be a fresh environment where neither resource exists, followed by a second deployment to check that it converges without changes. If that works, it would avoid reading the ID and replacing a placeholder in CI.

jlb

data_pulse
Contributor

@koosha 

I tested this end-to-end using Databricks CLI v1.22.1 with the direct deployment engine, and confirmed that a Genie Space and an AI/BI Dashboard can be deployed together in a single bundle deploy, without hardcoding IDs or requiring a second deployment.

The key is to use .id, not .space_id:

In yaml

overrideId: ${resources.genie_spaces.test_genie.id}

Important distinction: space_id does exist in the Genie API response, but ${resources.genie_spaces.test_genie.space_id} is not available as a bundle resource reference in CLI v1.22.1.

In my tests, .space_id passed bundle validate but failed during deployment planning with: 

field "space_id" not found in resources.GenieSpaceConfig

The bundle reference must use .id, even though the Genie API returns the identifier as space_id.

Another useful finding: Both approaches worked in my tests:

  • Inline serialized_dashboard, using structured YAML or a JSON string.
  • External .lvdash.json referenced through file_path.

So switching to inline serialized_dashboard isn't necessary with the tested CLI version.

Minimal configuration

In `databricks.yml`: 

bundle:
  name: genie-dashboard
  engine: direct

variables:
  warehouse_id: {}

resources:
  genie_spaces:
    test_genie:
      title: Genie test
      warehouse_id: ${var.warehouse_id}
      serialized_space:
        version: 2
        data_sources:
          tables:
            - identifier: samples.nyctaxi.trips

  dashboards:
    test_dashboard:
      display_name: Genie integration test
      warehouse_id: ${var.warehouse_id}
      file_path: ./dashboard.lvdash.json

In the existing valid dashboard.lvdash.json, configure: 

"uiSettings": {
  "genieSpace": {
    "isEnabled": true,
    "enablementMode": "ENABLED",
    "overrideId": "${resources.genie_spaces.test_genie.id}"
  }
}

What I am have validated:

  • First deployment: Both resources were created successfully in one deployment.
  • API verification: The dashboard's persisted overrideId exactly matched the newly created Genie Space's space_id.
  • Published dashboard  end-to-end validation: Asking Genie successfully returned data, and clicking Open Genie Agent opened the exact bundle created Genie Space, confirmed by its ID in the URL.
  • Dependency resolution: Genie was created before the dashboard without an explicit depends_on.
  • Redeployment: Resource IDs remained stable.
  • Second fresh target: The dashboard correctly referenced its own newly created Genie Space.
  • Dashboard update: The link remained correct after a definition change.

Earlier I was using CLI v0.290.0 and it did not expose genie_spaces in its bundle schema.

Conclusion: With CLI v1.22.1, the existing file_path approach supports automatic resolution of the Genie Space ID. There's no need for custom JSON templating or a two-stage deployment for this scenario.

Hope this helps and answers your question.

References:
- Databricks bundle resource reference
- Genie Space bundle resource types

Lu_Wang_ENB_DBX
Databricks Employee
Databricks Employee

This is a current DAB limitation. CLI 1.20 supports Genie spaces with the direct engine, but the documented dashboard resource has no Genie-space binding field. The dashboard JSON file also isnโ€™t a place where bundle resource references are resolved, so ${resources.genie_spaces.my_space.id} there wonโ€™t be substituted.

Workaround: make the pipeline two ordered, targeted stages: deploy the Genie space, get its ID from the bundle summary, render the dashboard JSON with that ID, then deploy the dashboard. That avoids the empty-ID bootstrap and a second full-bundle deploy, though it still requires two targeted deployment steps. Thereโ€™s no documented native way to make the dashboard depend on the Genie space ID today.