Construction and evaluation of SEO-based features for to use in learning to rank algorithms

The past work on information retrieval targeting web document collections shows that incorporating a measure that is solely based on documents (query-independent) and measures the quality of web documents, or rather the document prior (e.g., PageRank), into an information retrieval system improves the retrieval effectiveness. In this study, we introduce new document priors, inspired by Search Engine Optimization techniques. We also empirically investigate their effect by employing them as features in a learning to rank deployment. The experiments are performed on the two standard Web Information Retrieval test collections: the ClueWeb09 and the ClueWeb12 datasets, which include 500 and 733 million web documents, respectively. TREC and NTCIR query sets, which target those collections and contain a total of 1,204 queries, are used as query sets in the experiments. A strong baseline is formed by using standard features introduced in the previous works, with respect to which the effect of newly introduced features in this study is empirically compared. The experimental results reveal that the features introduced in this work led to statistically significant improvements in retrieval performance on the test collections in use (e.g., for the ClueWeb09 dataset, 18% improvement on average nDCG@10 score). The introduced features are classified into 5 groups with respect to functional properties and the contribution of each group to retrieval performance is also analyzed in detail.