Identical user matching on cross online social networks: A two-step approach based on face recognition

As the number of online social networks (OSNs) increases and the services distinct OSNs offer diversify, the number of individuals with accounts in more than one OSN increases. Consequently, the data of such users is spread over different OSNs. The problem of matching identical profiles on cross OSNs tries to unify OSN accounts of the same individual on different OSNs. Matching identical profiles not only allows having more information about a matched user, but also enables us to better understand users' behavior. Existing studies on matching identical profiles rely on profile attributes (such as age, gender, e-mail and education), content shared by users (such as tweets, photos, location check-ins) and links. However, relying on profile attributes is not reliable because any malicious user can easily spoof profile attributes to clone an OSN account. Furthermore, many users hide their profile attribute values due to privacy concerns, which bears the matching effort fruitless. Regarding user generated content, short and dirty texts within tweets are not easily processed, and location check-ins are much less frequently shared over many OSNs except Foursquare and alike. In this thesis, we propose a solution that relies on shared photos and network structure. Since face biometrics are quite personal and more distinctive features than any other alternative OSN feature, our solution provides an accurate and reliable solution towards matching identical users. Our novel solution has two-phases: the search phase and the match phase. The search phase inspects all photos shared by a user, extracts faces, identifies the profile owner's faces automatically and generates a face vector signature from a sample of these faces. Using signatures reduces algorithmic complexity of the process and also makes it more reliable. In the search phase, pairs of cross OSN accounts are compared over their signatures. Given an input source profile, the search phase acts as a filter to limit potential matches in the destination OSN to a small set of candidates. Subsequently, in the match phase, a ranked list of identical profiles are compiled using the rate of overlap between the links of the matched OSN profiles. Since there are many links of each account, we restrict the match phase to only profile photos of the link. Experimental evaluation is performed over 3.000 real-world, labelled OSN accounts from Facebook and Instagram. The results indicate that in the search phase, 89% of the given input source profiles are located over the destination OSN. 58% of them are in top-1 rank, 76% of them are in top-2 rank, and 87% of them are in top-4 rank. Morever, the match phase matches users with an f-score of 66%±4. Finally, it is shown that the proposed search phase provides faster and more accurate matching than existing solutions.