Performance improvement via distribited file systems on private analytic clouds

Improvements on current technologies and social media cause increase in data volume, variety, and velocity. Instant access to stored data and providing decision support is very hard with current technologies. Standard data base technologies cannot analyze the data and retrieve information from text data. Current solutions and methodologies have restricted analysis capabilities for Turkish texts. In this study a distributed analytical system and applications are developed for data bound and CPU bound applications. Performance improvements via distributed file systems are implemented on the system. The improvements are observed on single node and multi node systems. Faster results are obtained via distributed file systems on distributed analytical system. Different algorithms are evaluated in terms of performance and correctness for microblog text analytics on private distributed analytical system. Distributed scoring algorithm gives faster results than k-means clustering algorithm for microblog text analytics. The distributed algorithms developed for text analytics are implemented on single node and multi node systems and compared in terms of cluster performance. Memory constraints are observed on cluster performance and minimum memory requirement of the system is evaluated. Faster querying on big data is provided via the distributed analytical system. Generic and scalable storage layer is provided for applications. Distributed architecture usage is proposed for distributed analytical applications. Hardware can be used with maximum efficiency, and node replacement can be done at minimum time and minimum hardware cost with the scalability and automated node replacement features of distributed file systems. To sum up, it is observed that distributed file systems provide important performance improvements and improve efficiency for analytical operations on private analytical clouds.