A network pruning algorithm based on visual search task

Neural Network models have big sizes because of today's complex problems such as image recognition and classification even if not trained for a large number of classes. In the future, when we reach the strong AI which is a dream of a system that is the most capable intelligence on the earth, the models will have thousands of classes and giant sizes. So, reducing the number of classes and reducing the model's sizes without dropping the accuracy would be very handy when we need to accelerate the performance during recognition. In the literature, there are lots of pruning techniques but those focus on only reducing the sizes of the models. Recently The Goal Driven Pruning method has been proposed. In addition to reducing the size, The Goal Driven Pruning removes unnecessary classes from the model. That is, unlike the other studies in the literature, The Goal Driven method prunes a model not only for a hidden layer but also prunes the output layer according to a target class of interest, and fine tuning isn't applied after pruning. By doing so, a goal oriented and smaller size model is gained. This method is inspired by top-down attention mechanisms in the human visual system. This attention mechanism is based on reducing the sensitivity of the characteristics of distractors in the environment. For this purpose, The Goal Driven Pruning method prunes the irrelevant output classes and also prunes the hidden layers according to the target classes of interest. In this study, we elaborate more on the goal oriented pruning idea, investigate its applicability and propose the Global Goal Driven Pruning method, which evaluates the model as a whole on the contrary of Goal Driven Pruning. Moreover, via this study we try to fix some weaknesses of the Goal Driven method and have better accuracy performance. In order to assess our method's performance and compare with the Goal Driven technique, we applied our method on the same three models as in the previous Goal Driven study. The results showed that our method reduces the size of the models and the flop rate giving better results compared to the previous method. We also indicate future research possibilities regarding network optimizations based on visual search tasks.