Paraphrase identification in turkish using distributed representations of words and sentences

Paraphrase identification in natural language processing is the task of deciding whether two sentences have the same meaning. It is a central topic with applications in many tasks such as machine translation, automatic text summarisation and plagiarism detection. Recent studies in this field report successful results using word embeddings, which are vector representations of words learned from large corpora using neural networks. In this thesis, we apply word embedding methods to Turkish paraphrase identification problem, which has limited research. The objective is to test if any improvement can be achieved over rule based methods. The main idea is to predict the paraphrase relation between a pair of sentences based on the similarity score calculated from word embeddings. A new corpus is created from Turkish texts and word embeddings are learned using neural networks. Several sentence similarity methods that make use of these word embeddings are evaluated against existing Turkish paraphrase corpora. Sentence pairs with a higher similarity score are found more likely to have paraphrase relation as expected. The results are close to the state-of-the-art in Turkish paraphrase identification. In summary, application of word embeddings to Turkish paraphrase identification is found a useful approach. Suggestions for improvements and further research directions are provided.