This paper investigates the use of boosting as a meta-learner in Hyperparameter Optimization (HPO) and stacking ensemble. It aims to provide advice on the appropriate meta-learners for stacking ensemble in HPO, without the need for hyperparameter tuning. The paper proposes an implicit regularization in the classical boosting algorithm and a novel non-parametric stop criterion for boosting, specifically designed for HPO. The findings suggest that boosting exhibits competitive and promising predictive power as a stacking meta-learner in HPO.
Publication date: 2 Feb 2024
Project Page: http://www.aic.uniovi.es
Paper: https://arxiv.org/pdf/2402.01379
