- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-24-2024 07:45 AM - edited 12-24-2024 08:12 AM
Hi @jtberman , you need to make several configurations in variables in order to run the deployment via Azure DevOps pipeline. We had configured our project successfully using DAB to deploy via Azure DevOps pipeline. Following are variables(refer attached screenshot) that are required in order to execute the deployment(you can add values as per your configuration).
Inside your azure-pipelines.yml file, add following code to make it work:-
trigger:
branches:
include:
- development
- release
paths:
include:
- resources/*.*
- azure-pipelines.yml
- databricks.yml
exclude:
- src/*.*
variables:
- name: ENV
value: 'dev'
# trigger:
# - release
# - development
# Specify the operating system for the agent that runs on the Azure virtual
# machine for the build pipeline (known as the build agent). The virtual
# machine image in this example uses the Ubuntu 22.04 virtual machine
# image in the Azure Pipeline agent pool. See
# https://learn.microsoft.com/azure/devops/pipelines/agents/hosted#software
pool:
vmImage: ubuntu-latest
name: 'name-of-your-pool'
# Download the files from the designated branch in the remote Git repository
# onto the build agent.
steps:
- checkout: self
persistCredentials: true
clean: true
- task : UsePythonVersion@0
condition : false
inputs:
versionSpec: '3.10'
architecture: x64
- script: |
python -m pip install --upgrade pip
displayName: 'Install/Upgrade pip'
- script: |
sudo rm '/usr/local/bin/databricks'
curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sudo sh
displayName: 'Install Databriks CLI'
- script: |
databricks configure --host ${DATABRICKS_HOST} --debug
displayName: 'Configure Databricks secrets and host'
- script: |
databricks bundle validate -t $(BUNDLE_TARGET) --debug
displayName: 'Validate Databricks bundle'
- script: |
databricks bundle deploy -t $(BUNDLE_TARGET) --debug --force-lock --auto-approve
displayName: 'Deploy Databricks bundle'
Make changes if required else if your directory structure is following the DAB template then it should work fine and deployment will be successful on your respective workspace.
The above code will trigger the deployment if you change any yml file in codebase in development or release branch of your source control provider. We are using DevOps GIT as source control to manage our versioning.
The target variable used here refers to the value present in your databricks.yml file corresponding to dev target but you can deploy on any target provided you had configured that in databricks.yml.
Thanks!
Nitin