cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

What is the best to way to not deploy/run a workflow in production?

sergiopolimante
New Contributor II

I am building and MLOps architecture.
I do not want to deploy the training workflow to prod. 

My first approach was to selectively not deploy the workflow to prod, but this does not seem to be possible as in this thread:
https://community.databricks.com/t5/data-engineering/databricks-bundles-how-to-select-which-jobs-res...

Then, I thought of checking in the first task of training workflow if it is prod and then skipping the whole job at once, not just the first task. But it does not seem to be possible to skip a whole job, only specific tasks of it, as per this thread:
https://community.databricks.com/t5/data-engineering/gracefully-stop-a-job-based-on-condition/td-p/3...

The only approach left is to check all job tasks in the production environment and skip them, but it is not an elegant approach. 

Is there a better approach?

1 REPLY 1

Walter_C
Databricks Employee
Databricks Employee

 

  • Target Override Feature: You can use the target override feature to specify different configurations for different environments. However, this does not provide a direct way to exclude specific job resources.
  • Environment-Specific Folders: Another approach is to create environment-specific folders and include only the necessary resources for each environment. This requires maintaining separate configurations for each environment, which can be cumbersome.

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group