cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Automated CI code checks using workflows when PR is raised

Oliver_Angelil
Valued Contributor II

I'm familiar with Github Actions workflows to automate code checks whenever a PR is raised to a specified branch. For example for Python code, very useful is if unit tests (e.g. pytest), syntax (flake8), and code formatting (black formatter), type hinting (mypy) are run whenever a PR is raised or a branched is merged.

I am working in Databricks and my git provider is Azure Devops. I see the "Workflows" asset in Databricks - but not sure how to configure it in a way that could accomodate the described checks. Note I know how to write the .yaml file for Github Actions. Just not sure how to do the equivalent in Databricks.

Thank you very much in advance!

2 REPLIES 2

Priyag1
Honored Contributor II

In a typical software development workflow (e.g. Github flow), a feature branch is created based on the master branch for feature development. A notebook can be synced to the feature branch via Github integration. Or a notebook can be exported from Databrick workspace to your laptop and code changes are committed to the feature branch with git commands. When the development is ready for review, a Pull Request (PR) will be set up and the feature branch will be deployed to a staging environment for integration testing. Once tested and approved, the feature branch will be merged into the master branch. The master branch is always ready to be deployed to production environments.

Oliver_Angelil
Valued Contributor II

Thanks, although not quite the answer I was looking for. I know the github flow branching strategy well. My question is how to setup CI/CD workflows in Databricks to check code (run workflows) when PRs are raised. I'm using Databricks and Azure Devops as my git provider. I know very well how to do this with Github actions but I am not using Github.โ€‹

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group