Databricks Asset Bundles - High Level Diagrams Flow

[Coffee77]

Hi guys!

Working recently in fully understanding (and helping others...) Databricks Asset Bundles (DAB) and having fun creating some diagrams with DAB flow at high level. First one contains flow with a simple deployment in PROD and second one contains the same but considering a potential previous TEST environment. My question is about whether these diagrams are correct or partially correct and am i missing out something else? 🤔 I'd like to get feedback to leave as is, improve or fix while keeping them kind of "simple" 😉 Thanks!!

First High Level Diagram:

DAB High Level Diagram

Second high level diagram, including potential TEST environment:

DAB with Tests High Level Diagram

Thanks in advance!

[ManojkMohan · Accepted solution]

@Coffee77 

1. Bundle Validation / Dry-run Step:
   Add a step before deployment to validate the bundle using commands like databricks bundle validate
2. Artifact Build Step (if relevant): If your repository includes built artifacts such as Python wheels or JAR files, include an explicit build step
3. Parameterization : Highlight secure handling and injection of secrets or runtime parameters via environment variables.
4. Error Handling / Rollback: Add a visual for failure scenarios with logging, notification,

[Coffee77]

Interesting, I think your suggestion can be indeed added while still keeping simple enough. I'll incorporate, thanks!

[Coffee77]

By the way, (I think) I fully understand your  first 3 points. Concerning 4th one, I know how to include tracing, logs, etc BUT how or where could I get guidance or documentation or  best practices for rollback actions. It is not an easy point, right?

[ManojkMohan]

@Coffee77 

Databricks deployment best practices and rollback: https://docs.databricks.com/dev-tools/databricks-cli.html

Backup and restore practices: https://docs.databricks.com/administration-guide/account-settings/workspace-backups.html

Schedule regular exports to safe locations and test restores docs.databricks.com best practices

[Coffee77]

Updated first high-level diagram. Now, it looks like this way:

DAB High Level Diagram v1.1

[Coffee77]

I will go only with latest version then 😉, that can be applied to any other lower environment for QA or testing.