Fısıltı tabanlı dağıtık derin öğrenme yönteminin geliştirilmesi

In this study we have studied fully-decentralized learning which is based on exchanging trained local models randomly using gossip protocols between nodes. Unlike other partially decentralized learning methods, in this approach there is no leader or coordinator node. The same algorithm executes in each node. Every node averages received models and trains the averaged model with its local data. Although it is theoretically not proved, the method works quite well in practice. In order to understand the algorithm, it is necessary to understand the underlying communication protocol used to distribute trained models. Gossip learning originally proposed based on push type gossip protocol. We have observed that there is little amount of study in this field. Since improvement of decentralized learning strongly depends on application of underlying protocols in an appropriate way and model merging strategies. In the first part of this research, we have studied all gossip protocols extensively both in SI and SIR models theoretically and empirically. We have derived numerous equations such as predicting peak time when network load reaches its maximum value in push type gossip in the SIR model. In most existing research, gossip algorithms are used to train linear models such as SVM and logistic regression. In the second part of this study, we have used neural networks for training on distributed data using gossip learning algorithm with various partial model transfer strategies for different network sizes.