topic Databricks Asset bundles in Administration & Architecture

Databricks Asset bundles

sri840 — Wed, 12 Feb 2025 10:48:01 GMT

Hi Team,

In our company we are planning to migrate our workflows with Databricks Asset bundles, is it mandatory to install Databricks CLI tool for getting started with DAB ? Any one who integrated with Github with CICD pipeline please let me know the process for getting started with Databricks asset bundles ?

Thanks

Sri

Re: Databricks Asset bundles

Stefan-Koch — Wed, 12 Feb 2025 11:58:28 GMT

Hi Sri

Yes, you should install Databricks CLI, because DAB's are part of the CLI.

See the doc for DAB: https://docs.databricks.com/en/dev-tools/bundles/index.html

For CI/CD-Pipeline with Github Action, see this: https://docs.databricks.com/en/dev-tools/bundles/ci-cd-bundles.html

Re: Databricks Asset bundles

-werners- — Fri, 14 Feb 2025 13:34:58 GMT

ok, if you are going to do that, there are a few things to know.
f.e. up to now you always have to define your spark version in the dab, even though the spark version is defined in a compute policy. So if you switch to a newer databricks version you have to do a redeploy of all your DABs.
(now this is not because of DAB itself but because of the databricks cli that has spark version as mandatory)
So definitely check out variables, substitutions and include.
https://docs.databricks.com/en/dev-tools/bundles/settings.html

Mind that the docs are not complete.

Re: Databricks Asset bundles

-werners- — Fri, 14 Feb 2025 13:40:33 GMT

I forgot the CI/CD part:
that is not that hard. Basically in DAB you define the type of environment you are using.
If you use 'development', DAB assumes you are in actual development mode (feature branch). so there you can connect git and put the files in a feature branch.
Deploying can be done manually.
Now, from there on, I suggest you use automated pipelines to promote to QA/PROD.
The pipelines put the files in the correct branches in git (using merge) and start the deploy of the dab on the correct environment.
For those (QA, PROD) use environment type 'production'.

So basically it comes down to making sure your DAB files are put into git in the correct branch and deploying in the correct workspace.