Analysis and evaluation of big data clusters in different cloud computing environments

Cloud technology has become the supreme way to deliver enterprise applications and the preferred solution for companies launching innovations or extending their infrastructure. NoSQL databases are often used to provide availability and flexibility for big data processing. However, in distributed and parallel processing environments, there has not yet been a comprehensive evaluation of NoSQL databases. NoSQL was developed to better meet the data storage needs of a large number of records, but there are different kinds of NoSQL technologies, and most have not been thoroughly compared. Understanding the performance of NoSQL databases is essential for the selection of the correct database for a specific application. Several databases have emerged that can handle the cloud domain to take advantage of the flexibility of the cloud environment. The aim of this dissertation is to develop a model to evaluate and analyze NoSQL databases using a cloud environment where these systems are deployed (i.e., in a cloud or a local data center cluster). This model is then evaluated with the most popular NoSQL databases in cloud computing (MongoDB and Riak KV) and internal cluster environments to analyze the performance of those two systems. A number of aspects of performance, including monitoring throughput and latency, are compared. The model architecture has the ability to use other databases as well as other clouds. An evaluation of the model system in three cloud computing infrastructures is also presented. The model architecture can use other databases as well as other clouds. Furthermore, a decision tree is constructed based on many experiments to determine the performance criteria for the databases on the selected cloud computing platforms. The main findings of this work are as follows: (i) Performance gain is achieved by integrating MongoDB with Google Cloud, especially in terms of reading and writing a massive amount of data in a cluster environment; (ii) The infrastructure of Google Cloud is more manageable and efficient in dealing with the distributed environment and big data, and it achieved better results than the other clouds; (iii) The results also showed the limitations of Riak KV in both the Google and OpenStack clouds, especially in terms of update latency. Keywords: NoSQL; cloud computing; MongoDB; Riak KV; big data; cluster; YCSB; Basho-bench