cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Is there a way to use name prefixes for asset bundle deployments to prevent collisions

Mathias_Peters
Contributor

Hi, 

we are using asset bundles to deploy artefacts in Databricks. We use a service account and github workflows for that. Currently, we deploy in dev mode whenever a PR is created against the main branch of our gh repo. So far so good. 

However, we are getting collisions when having multiple open PRs. That leads to overriding and worse deleting each others changes when creating/updating PRs. 

Is there a way to prevent resources from being undeployed/overwritten in this scenario?

I am looking into adding the github branch name into the name prefix but I am unsure if this fixes the problem. 

 

1 ACCEPTED SOLUTION

Accepted Solutions

filipniziol
Contributor III

Hi @Mathias_Peters ,

Yes, it is possible to do so.

1. Configure variable that will be added to the path.

variables:
  feature_name:
    description: This will be added to the path

2. Configure workspace root_path:

workspace:
   root_path: /Shared/.bundle/$(var.feature_name}/${bundle.name}

3. Run deployment from your CI/CD pipeline providing feature_name variable:

databricks bundle deploy --var="feature_name=my_feature"

4. Check the article on varriables in Databricks Asset bundles:
https://docs.databricks.com/en/dev-tools/bundles/variables.html

View solution in original post

1 REPLY 1

filipniziol
Contributor III

Hi @Mathias_Peters ,

Yes, it is possible to do so.

1. Configure variable that will be added to the path.

variables:
  feature_name:
    description: This will be added to the path

2. Configure workspace root_path:

workspace:
   root_path: /Shared/.bundle/$(var.feature_name}/${bundle.name}

3. Run deployment from your CI/CD pipeline providing feature_name variable:

databricks bundle deploy --var="feature_name=my_feature"

4. Check the article on varriables in Databricks Asset bundles:
https://docs.databricks.com/en/dev-tools/bundles/variables.html

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group