cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

mlops-stacks workflow testing vs staging

MohsenJ
Contributor

I'm a newbie to MLOps and abit confused about the use and the implementation of staging and testing environment in the mlops-stack template. 

as far as I understand the staging environment is where we run the integration test. But in the ci-cd pipeline the integration_test is deployed and run the test target. And in the staging environment it deploys the all the workflow without running them. why is that? I'd appreciate any explanation. 

 

 

1 REPLY 1

Dennisleon
New Contributor II

@MohsenJOfficial Site wrote:

I'm a newbie to MLOps and abit confused about the use and the implementation of staging and testing environment in the mlops-stack template. 

as far as I understand the staging environment is where we run the integration test. But in the ci-cd pipeline the integration_test is deployed and run the test target. And in the staging environment it deploys the all the workflow without running them. why is that? I'd appreciate any explanation. 

 

 


Hello,

You're right, there's a subtle difference between how MLOps Stacks handles testing vs staging environments. Here's the breakdown:

Testing Environment: This is where you run unit tests and potentially some basic integration tests. These tests focus on individual components of your ML workflow (data processing, model training, etc.) in isolation. The CI/CD pipeline typically deploys the code for these tests and executes them directly.

Staging Environment: This environment mimics production as closely as possible. Here, you deploy the entire ML workflow but might not run all the steps automatically. Instead, the focus is on manually testing the integration between all components and ensuring everything works together seamlessly before pushing to production.

This separation allows for efficient testing:

Unit and basic integration tests are faster to run in the CI/CD pipeline, providing quicker feedback.
Staging lets you thoroughly test complex interactions and real-world data scenarios before risking production issues.

I hope the information may helps you. 

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group