Databricks dashboards across multiple Databricks instances

thisisthemurph — Fri, 20 Dec 2024 09:47:43 GMT

We have multiple Databricks instances, one per environment (Dev-UK, Live-UK Live-EU, Live-US, etc), and we would like to create dashboards to present stats on our data in each of these environments. Each of these environments also has a differently named catalogs, such as `lake_development` and `lake_live`, so queries that back the dashboard data would be slightly different depending on the environment.

I was hoping there would be a way to create a Dashboard in code (preferably in Python) and deploy this from our repo, similar to how we deploy our Databricks jobs with CI. Currently, I believe I would have to manually create these dashboards in each of the environments, which would make maintaining them a difficult and time consuming task.

Does anyone know of a good method to implement a dashboard across different Databricks instances like this?

I have seen that there is an API endpoint for creating dashboards, so this is a possibility. There also seems to be a `dashboards` function in the CLI, though the documentation on the help text for the CLI states:

Databricks SQL alerts The alerts API can be used to perform CRUD operations on alerts. dashboards In general, there is little need to modify dashboards using the API.

Re: Databricks dashboards across multiple Databricks instances

Walter_C — Fri, 20 Dec 2024 11:40:41 GMT

Hello, as you have mentioned you could create an script in Python that uses the api call https://docs.databricks.com/api/workspace/lakeview/create to generate the dashboard for each environment, the process to create the visualizations will be complex but it is indeed posible.

topic Re: Databricks dashboards across multiple Databricks instances in Data Engineering

Databricks dashboards across multiple Databricks instances

Re: Databricks dashboards across multiple Databricks instances