A distributed human identification system for indoor environments

Nowadays "internet of things" and "robotics" topics are getting more and more popular because "state of the art" technologies are easier to be applied in our daily lives. Utilization of "internet of things" started home automation systems to use devices at home remotely. While these automation systems aim at giving access to designated areas, they also offer solutions for unauthorized access problems by keeping logs of video records and control mechanisms as well as features for more security measures. Besides, developers are able to implement and monitor video surveillance systems to support security in the IoT platforms using low cost embedded computers. This study presents an entire system, which is similar to home automation system and allows the login of a person entering a private property using face recognition system, video calls, sending notifications and having more components such as threat detection. In this study statically positioned camera systems were implemented to identify people thanks to face recognition algorithms. Captured images from fixed positioned camera systems(FPFRS) had problems to process on identification. The problems were differences in distance of people to the camera and differences caused by posing. Therefore a mobile robot that has camera, was incorporated to recognize faces of people in the environment. The mobile robot can track people and keep the distance to the people constant. Thus, the mobile robot can solve the distance problem. However, the mobile robot can not track people perfectly without miss. Sometimes it can have trouble while tracking people. Thus, integrating the mobile robot to the Fixed Positioned Face Recognition System has a big potential to deal with the face recognition problem in indoor environments. Therefore an distributed approach was proposed to recognize humans in this study. The system utilizes 1) Cameras statically mounted on wall devices (FPFRS), 2) Cameras mounted on mobile robots. FPFRS were built on a tiny embedded computer. Motion sensor and infrared camera components were used to gather data about environment. However, sometimes they can fail to recognize faces because of their fixed positions. This decreases face recognition performance of the system when FPFRS tries to recognize person from captured images. Because distance between camera and person is very important in order to obtain features of face. To improve the performance of this system proposed method given in this thesis aims at solving the problem of fixed position with robots. Therefore a mobile robot is added to improve face recognition accuracy gathering face images from certain distance and moving around face image that is aligned to the camera of robot before captured. With the help of the mobile robot, the system becomes more interactive and information gathering from environment process is done more efficiently. As a result proposed hybrid system in this study not only uses fixed positioned captured images by FPFRS but also uses images that are taken from mobile robot. To provide network communication between FPFRS and the mobile robot which are linked to in this distributed system, modified techniques are used which follow the rules of client-server approach. Each of the FPFRS and the mobile robot runs main program during scenarios. Inputs can be taken from the environment and enable the trigger mechanism. An FPFRS schedules its jobs and behaves according to the active jobs. After the job is done, FPFRS deactivates the jobs in the database records. These records can be manipulated from other devices. Thus communication and trigger mechanism between devices is established. These systems are more complex and it is hard to follow this kind of systems, actions. Architecture is designed to expand by adding new FPFRSs or mobile robots to the system and each FPFRS can communicate via server. One FPFRS and a mobile robot have scenario to capture more images of people from different positions. Then each device runs face recognizer and obtains a result which consists of name of person and confidence of face recognition score. Using the results of face recognizer, various techniques, including machine learning algorithms, are applied to improve accuracy of face recognition. In addition to choosing prediction based on highest confidence in face recognition, random forest tree, decision tree, and linear SVM methods are applied to address the issue of face recognizer improvement. Described systems are implemented and tested systematically based on various scenarios. To show the impact of the effect of changes on accuracy of face recognition system, experiments were done with and without the mobile robot. The proposed system was evaluated using 1)Face recognition scores on a face database, 2)Performance of video calls, and 3)Case scenarios performed on human test subjects. We have verified the effectivity of our approach by using six scenarios. In the first scenario button triggered human identification has been evaluated. In the second scenario motion triggered human identification has been analyzed. In the third scenario human identification using rotational tracking has been investigated. In the fourth scenario human identification using displacement tracking has been examined. In the fifth scenario human identification using displacement and voice combined tracking has been illustrated. In the sixth scenario human identification using robot initiated dialog has been analyzed. One or more FPFRS and the mobile robot have scenario to route the mobile robot to capture images of people on designated area of motion sensor triggered FPFRS based on the scenarios given above.