Privacy preserving anonymization of big data and data streams

Traditional data anonymization methods have been developed only for static datasets, where the scalability has usually been disregarded. With the diversified increase of big data and streaming data needs in recent years, the scalability and dynamic nature of data started to come to the foreground. Although studies have been proposed in the literature to provide big data and streaming data privacy solutions, more effective and high coverage data anonymization methods are needed due to various traits of the problem. Within the scope of this thesis, more effective and high coverage anonymization methods have been studied to ensure big data and streaming data privacy. Apache Spark is among the most advanced technologies and platforms in the field of big data processing. In this thesis, a distributed big data k-anonymization method is proposed, which takes big data anonymization as a special case of big data processing and uses the top-down specialization search technique on the domain hierarchy of quasi-identifier attributes. Information gain - privacy loss metric is used as the search criteria. The effectiveness and the scalability of the method have been demonstrated on extended real datasets. The solutions developed for k-anonymization of data streams in the literature are low coverage solutions that formulate the problem as a single-objective optimization problem that tries to minimize the information loss metric on quasi-identifier attributes. High coverage solutions have been proposed for the needs identified within the scope of the thesis and their effectiveness on real data sets has been shown through extensive experimental evaluations. First, a multi-objective optimization framework is proposed to minimize the information loss and average delay together for streaming data. Thus, the data utility for streaming data is measured as a function of the data quality measured by the information loss metric and the data aging measured by the average delay metric. In the proposed method, the component weights can be tuned by the user. Moreover, a custom information loss metric is introduced. Secondly, a down-stream data analysis process aware k-anonymization framework is proposed. Many data recipients are known to run classification data mining tasks on the anonymized data. Therefore, in this study, besides minimizing information loss, maximizing classification accuracy is another objective. In fact, in case there exists sensitive attributes in addition to the quasi-identifier and the classification target attributes, the sensitivity of these sensitive attributes should be maintained at the highest level. The proposed method solves this three-objective optimization problem, the weights of which can be tuned by the user.