Optimal Ensembles for Deep Learning: Theory and Practice

Li, Wenjing

Etd

Optimal Ensembles for Deep Learning: Theory and Practice

Public

Ensemble learning is a process by which multiple base learners are strategically generated and combined into one composite learner. There are two features that are essential to an ensemble’s performance, namely the individual accuracies of the component base learners and the overall diversity level of the ensemble. It is evident that diversity must have a role to play in the effectiveness of an ensemble, since multiple identical copies of the same learner clearly cannot improve on the performance of a single copy of that learner. Ideally a perfect ensemble would have both accurate and diverse component learners. However, this is impossible as there is always a trade-off between learner accuracy and diversity in an ensemble. One of our major contributions is that we derived a theory for both classification and regression contexts that can rigorously bound the relationship between learner accuracy and diversity in ensembles from the perspective of statistical correlations. Then the theory we proposed has inspired a methodology for assessing and improving the optimality/performance of any given ensemble, including ensembles of deep neural networks.

Creator