Connectivity optimization of artificial neural network with evolutionary computation algorithms

Optimization is the process of finding the most suitable solution among possible solutions for a problem. Optimization problems aim at maximizing profit or minimizing cost. Optimization methods can generally be considered in two categories as classical and heuristic methods. Due to the large number of parameters and the large search space, classical optimization methods are insufficient in solving real-world problems. In such cases, heuristic optimization methods are preferred. Heuristic optimization methods use natural phenomena to reach a solution to achieve a goal. The use of natural phenomena reveals the concept of herd intelligence. Swarm intelligence is the collective behavior of self-organizing and decentralized systems. Flock of birds, fish, ant colonies etc. to the herd intelligence found in nature. Animals that live in flocks are examples. Particle Swarm Optimization (PSO), which is inspired by the movements of living things in nature, is also a swarm intelligence algorithm. The main types of networks similar to biological neurons are Hopfield, Boltzman machines, repetitive networks, and needle networks. Although the structure of these networks is mostly random, they can also be created manually. Thanks to the advancement of technology day by day, these approaches that model biological nervous systems have been paved the way. In this study, a new network architecture was developed, inspired by biological nerve cells. In order for the artificial neural network to be created to give the best result, the connections in the network were considered as a binary problem and a solution was sought with BPSO (Binary Particle Swarm Optimization). In addition, the weights of the connections in the artificial neural network are continuously optimized with PSO. As a result, a flexible and highly accurate network structure has been created that is compatible with the biological nervous system and can be applied to different types of problems. This network structure was applied on different data sets and compared with fully connected artificial neural networks whose weights were trained with both PSO and BP (Backpropagation). In addition, the connection optimization process with GA (Genetic Algorithm), which is one of the most preferred evolutionary computation methods for the connection optimization of the artificial neural network in the literature, is also compared with the proposed method. As a result, a swarm intelligence algorithm has been proposed for an artificial neural network design that is more suitable for the structure of biological neural networks. The performance of the proposed method was investigated on 4 data sets and comparisons were made. The results show that neural network connectivity optimization improves the classification performance of the network.