I think I figured it out. Autologing must have been enabled from a previous run. Pretty easy to solve. Posting this to help anyone else in this situation.
https://mlflow.org/docs/latest/tracking.html#automatic-logging
To disable just run the appropriate command for the library being logged as per below.
import mlflow
mlflow.sklearn.autolog(disable=True)
mlflow.xgboost.autolog(disable=True)
mlflow.statsmodels.autolog(disable=True)