Automatic video indexing and retrieval with skin color based face detection method

Ill ABSTRACT This thesis covers face detection, face tracking, facial expressions, video indexing and retrieval subjects and presents a system for automatically detecting, extracting, indexing and retrieval of human facial information from raw video. In addition, it also introduces a new object motion based shot detection method. Objects are tracked and their locations are saved to detect video shots boundaries. If the object changes its previous location then a shot is detected. The system extracts human face information from video frames and then locates facial features of the face. Time required to extract human facial information is about 0.33 seconds (3 fps) in average. Each of the face information in each frame is stored in a database of video frames. The user is then either uses the pre-extracted information in retrieval of faces or defines trajectories of human face objects in order to retrieve specific types of actions performed by the face. The system developed using Borland C++ Builder 5.0 and can operate on AVI and MPEG files as well as real-time live video feeds from video camera. Keywords: Video indexing, video retrieval, computer vision, face detection, face tracking, facial expressions