Rie Johnson , RJ Research Consulting, Tarrytown
Tong Zhang , Rutgers University
We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner as a black box, the method we propose directly learns decision forests via fully-corrective regularized greedy search using the underlying forest structure. Our method achieves higher accuracy and smaller models than gradient boosting on many of the datasets we have tested on.
Boosting, Decision trees, Vegetation, Additives, Tuning, Greedy algorithms, Vectors, greedy algorithm, boosting, decision tree, decision forest, ensemble
T. Zhang and R. Johnson, "Learning Nonlinear Functions Using Regularized Greedy Forest," in IEEE Transactions on Pattern Analysis & Machine Intelligence.