cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Avoid Using MLFlow to log runs

Evan_MCK
Contributor

I am running some code to smooth columns of data and fill in missing values. The algorithms being used are automatically logged to an experiment using MLFlow which makes a 15 second job take 9 minutes and causes out of memory errors in another instance. I have not even imported the MLFlow package in the notebook. How do I avoid logging results?

1 ACCEPTED SOLUTION

Accepted Solutions

Evan_MCK
Contributor

I think I figured it out. Autologing must have been enabled from a previous run. Pretty easy to solve. Posting this to help anyone else in this situation.

https://mlflow.org/docs/latest/tracking.html#automatic-logging

To disable just run the appropriate command for the library being logged as per below.

import mlflow

mlflow.sklearn.autolog(disable=True)

mlflow.xgboost.autolog(disable=True)

mlflow.statsmodels.autolog(disable=True)

View solution in original post

2 REPLIES 2

Evan_MCK
Contributor

I think I figured it out. Autologing must have been enabled from a previous run. Pretty easy to solve. Posting this to help anyone else in this situation.

https://mlflow.org/docs/latest/tracking.html#automatic-logging

To disable just run the appropriate command for the library being logged as per below.

import mlflow

mlflow.sklearn.autolog(disable=True)

mlflow.xgboost.autolog(disable=True)

mlflow.statsmodels.autolog(disable=True)

Evan_MCK
Contributor

You can even us a general command without noting which library will be used:

import mlflow
mlflow.autolog(disable=True)
Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.