Implementing machine learning applications using distributed data management and processing architecture

With the development of technology that takes place more and more every day in our lives, it becomes almost impossible to manage and process the data produced and thus brought about the necessity of storage and analysis. Both the data size and the increase in the variety of data have necessitated the development of new methods in this context. In this thesis, machine learning applications have been developed by using distributed data management and analysis tools which have been developed for data that cannot be processed in traditional management. Applications were implemented using pyspark libraries on the Spark cluster created using the Google Cloud service. In this thesis, machine learning applications were carried out by using two different data sets. The application of machine learning was developed with Logistic Regression classification algorithm by using motion data obtained from wireless sensors in application-1. The use of resources in the cluster was observed during the execution of the application. In the application-2, the average clicks cost estimation performances of Random Forest and Gradient-boosted Tree algorithms were compared by using the data obtained from the control panel of an online tourism agency.