Construction and performance analysis of locally adaptive base and ensemble learners

In this study, construction and performance analysis of locally adaptive base and ensemble learners have been proposed by using Meta and Ensemble Learning techniques. The characteristics and meta-features of the discretized sub regions in a dataset have been analyzed for the purpose of better learning performance. A detailed performance analysis of a local base learner over any type of dataset is firstly performed in order to understand the reasons of both failure and success in classification. Prediction of the performance of a base learner (e.g., decision tree) is proposed with a feasible linear regression model by examining the local geometrical complexities. As a consequence, some meaningful reasons have been determined why decision trees outperform or fail on any dataset. Additionally, the discrete sub regions are learned by using the Mixture of Experts model to enhance the overall prediction accuracy. A new approach in Mixture of Experts using hard clustering techniques is presented with different types of gate functions for accurate prediction and classification. Finally, a localized lazy base learner using a dynamic parameter selection mechanism is established to gain better performance. A locally adaptive parameter selection mechanism for nearest neighbor classifiers using clustering methods is suggested. In the experiments of the improved method more accurate results have been found.