Summary
Ensemble Learning
So we have created multiple models using ANN, CNN, TL and these models have various accuracies and F1-Scores. To best use the features of each model we use 'Ensemble Learning'! In the world of Statistics and Machine Learning, Ensemble learning techniques attempt to make the performance of the predictive models better by improving their accuracy. Ensemble Learning is a process using which multiple machine learning models (such as classifiers) are strategically constructed to solve a particular problem. They combine the decisions from multiple models to improve the overall performance. There are three ways Max Voting, Averaging, Weighted Averaging. The max voting method is generally used for classification problems. In this technique, multiple models are used to make predictions for each data point. The predictions by each model are considered as a ‘vote’. The predictions which we get from the majority of the models are used as the final prediction. Similar to the max voting technique, multiple predictions are made for each data point in averaging. In this method, we take an average of predictions from all the models and use it to make the final prediction. Averaging can be used for making predictions in regression problems or while calculating probabilities for classification problems. This is an extension of the averaging method. All models are assigned different weights defining the importance of each model for prediction.
An ensemble is itself a supervised learning algorithm, because it can be trained and then used to make predictions. The trained ensemble, therefore, represents a single hypothesis. This hypothesis, however, is not necessarily contained within the hypothesis space of the models from which it is built. Thus, ensembles can be shown to have more flexibility in the functions they can represent. This flexibility can, in theory, enable them to over-fit the training data more than a single model would, but in practice, some ensemble techniques (especially bagging) tend to reduce problems related to over-fitting of the training data.
Bagging : Bagging is a hugely popular ensembling method which is used in algorithms like Random Forest. It gains accuracy by not only averaging the models but also trying to create models that are as uncorrelated as possible by giving them different training sets. It creates the data-set using sampling with replacement a simple but sufficient data sampling technique. To implement this we will create a method called subsample and call it for every model to create its individual data-set.
Boosting: Boosting is a sequential process, where each subsequent model tries to correct the errors of the previous model. Therefore the succeeding models are dependent on the previous models and we need to train the models in sequence instead of parallel.