Clustering-based selective neural network ensemble

INTRODUCTION Neural network ensemble is becoming a hot spot in machine learning and data mining recently. Many researchers have shown that simply combining the output of many neural networks can generate more accurate predictions than that of any of the individual networks. Most previous work either focused on how to combine the output of multiple trained networks or how to directly design a good set of neural networks. Theoretical and empirical work showed that a good ensemble is one wh…

Clustering-based selective neural network ensemble

An effective ensemble should consist of a set of networks that are both accurate and diverse. We propose a novel clustering-based selective algorithm for constructing neural network ensemble, where clustering technology is used to classify trained networks according to similarity and optimally select the most accurate individual network from each cluster to make up the ensemble. Empirical studies on regression of four typical datasets showed that this approach yields significantly smaller en semble achieving better performance than other traditional ones such as Bagging and Boosting. The bias variance decomposition of the predictive error shows that the success of the proposed approach may lie in its properly tuning the bias/variance trade-offto reduce the prediction error (the sum of bias2 and variance).