topic Re: Is there a way to use name prefixes for asset bundle deployments to prevent collisions in Data Engineering

Is there a way to use name prefixes for asset bundle deployments to prevent collisions

Mathias_Peters — Wed, 09 Oct 2024 15:58:42 GMT

Hi,

we are using asset bundles to deploy artefacts in Databricks. We use a service account and github workflows for that. Currently, we deploy in dev mode whenever a PR is created against the main branch of our gh repo. So far so good.

However, we are getting collisions when having multiple open PRs. That leads to overriding and worse deleting each others changes when creating/updating PRs.

Is there a way to prevent resources from being undeployed/overwritten in this scenario?

I am looking into adding the github branch name into the name prefix but I am unsure if this fixes the problem.

Re: Is there a way to use name prefixes for asset bundle deployments to prevent collisions

filipniziol — Wed, 09 Oct 2024 19:00:32 GMT

Hi @Mathias_Peters ,

Yes, it is possible to do so.

1. Configure variable that will be added to the path.

variables: feature_name: description: This will be added to the path

2. Configure workspace root_path:

workspace: root_path: /Shared/.bundle/$(var.feature_name}/${bundle.name}

3. Run deployment from your CI/CD pipeline providing feature_name variable:

databricks bundle deploy --var="feature_name=my_feature"

4. Check the article on varriables in Databricks Asset bundles:
https://docs.databricks.com/en/dev-tools/bundles/variables.html