I use LightGBM model and it's method train. And there is a parameter verbose_eval=1 that prints LightGBM's progress.

lgb.train(params,dataset, verbose_eval=1)

I want to catch this progress (save in variable): I start train model from Flask, so I need to send this progress through REST API.

Is it possible?

P.S. One thing making my problem harder: I use multiprocessing module (Process). I run several models simultaniously. So I can't use @Mykhailo Lisovyi approach: models will print into one flow. I can not determine which data from which model.

1 Answers

Mykhailo Lisovyi On

You catch catch stdout and stderr directly. I used the solution from this gist. I'm not experienced with the native training API. But using the sklearn API of lightgbm you can do:

import sys
from io import StringIO

def capture_out(func):
        old_stdout = sys.stdout
        result = StringIO()
        sys.stdout = result
        result_string = result.getvalue()
        return result_string
        sys.stdout = old_stdout

def fit_model():
    mdl.fit(train_X, train_y, **params_fit)


where mdl is lgb.LGBMRegressor, train_X, train_y are numpy arrays, params_fit are parameters to configure early stopping