Implementation of distributed RDFS reasoning

We live in big data age in which many computational tasks either generate or need to use large datasets. This makes parallel and distributed computing a key for scalability. MapReduce is a programming model for processing large datasets in parallel and distributed fashion on cluster of computers. Today, since the size and complexity of RDFS documents increase rapidly, RDFS reasoning problem has to embrace and address the big data solutions. The output of RDFS reasoning job can be input to another job and the output of RDFS reasoning jobs grow big as the input documents gets bigger. In this study, MapReduce programming model, in particular Hadoop with related core technology like Hive and Pig, is used for improving the performance of distributed RDFS reasoning. Additionally, document indexing is used for further performance gain. The study shows that both of the MapReduce and indexing approaches are quite effective and offer significant performance improvements and scalable solutions. Experimental evaluations on two RDFS datasets, namely Dbpedia and Freebase, are provided.