Design and implementation of a response retrieval and reranking system

In this master's thesis, the work has been started with a baseline response retrieval and re-ranking system composed of two steps: BM25 retrieval and BERT re-ranking. After investigating the effects of several parameters and BERT model size on the baseline approach, a unique retrieval and re-ranking system with TF-IDF retrieval and Cross-Encoder re-ranking steps has been designed and implemented. With the application of Deep Learning models to the re-ranking step, consistent ranking performance improvements have been observed. The research focus of this thesis has been selected as the comparative performance study of different Transformer models. With smaller models that can be trained and evaluated faster and more efficiently, equal or at least relatively competitive results have been targeted. It has been shown that smaller transformer models can even out-perform larger models. Additionally, this designed re-ranking system has been re-purposed for a Question Answering task where the answer for a given question is searched as a subset of a passage. Even though the re-ranking system has been directly used without undergoing any modifications regarding the QA task, promising results that are worth further research have been attained.