Hi @tim-mcwilliams, @haseebasif, @Mickel, @g000gl, @Piotrus321,
I encountered a similar problem with a Prophet Markov chain Monte Carlo (MCMC) model that caused my browser to completely drain both the CPU and RAM. Even after the workflow was completed, attempting to open either a notebook or a script used to run the code resulted in the same issue.
I suspect the behaviour you are experiencing is similar to mine and is related to the large amount of log output generated during the Bayesian inference process.
In my case, this behaviour was caused by the cmdstanpy library which is what runs the MCMC simulation and likely stems from the underlying C++ compiled code. Therefore, using the logging library to set different levels for the python packages does not work.
Since the output is printed out directly to the stdout and stderr streams by the underlying Stan processes, regardless of whether you are running the code from a notebook or a script file, you can just redirect both stdout and stderr streams while running the code with a custom contextmanager class.
import os
import sys
from contextlib import contextmanager
@contextmanager
def suppress_stdout_stderr():
with open(os.devnull, 'w') as devnull:
old_stdout = sys.stdout
old_stderr = sys.stderr
try:
sys.stdout = devnull
sys.stderr = devnull
yield
finally:
sys.stdout = old_stdout
sys.stderr = old_stderr
Then simply wrap your training/fitting code with the method:
with suppress_stdout_stderr():
model = your_model_class()
model.fit(your_data)This way the context manager opens a null file for writing the logs and then discards all data written to it.
