dmlc / treelite

Universal model exchange and serialization format for decision tree forests

Home Page:https://treelite.readthedocs.io/en/latest/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support boosting from the average in sckit-learn

hcho3 opened this issue · comments

Scikit-learn's gradient boosting algorithm performs "boosting from the average," where a simple base estimator is fitted from the distribution of class labels (or the average label, if regression is used) and is set as the initial learner in the ensemble model. Boosting from the average speeds up convergence. See dmlc/xgboost#4321 and https://lightgbm.readthedocs.io/en/latest/Parameters.html#boost_from_average

Currently, Treelite throws an error if init="zero" is omitted when building GradientBoostingClassifier / GradientBoostingRegressor objects. We should remove this restriction, in order to support boosting from the average