Natural scene image text detection and recognition using a novel global curvature feature

This thesis deals with scene text detection and recognition as a multiple object detection and recognition problem. That is, texts that are buried within an image naturally are detected and recognized character by character. As a result, the recognition process is usually termed as segmented or cropped character recognition. Two approaches for scene character detection are introduced. The first one is clustering based segmentation technique for multi-color scene text detection. This approach is designed to scene image texts, especially with intra-word color variance. That is, characters within the same word have distinct colors. The second approach is inspired by Maximally Stable Extremal Regions (MSER) for connected component generation. However, in this thesis, instead of stable regions, unstable regions are considered to generate candidate characters. The approach is termed as Maximally Unstable Extremal Regions (MUER) throughout the thesis. For cropped scene character recognition, a classical approach for general object recognition is employed. That is, descriptive image features are hand-engineered and are used to train a supervised learning algorithm for recognition. Therefore, a keypoint detection and description strategy is introduced to describe the shape of character images globally. Curvature information is the primary geometric property that is employed to identify qualified keypoints. The description is dependent on major properties such as physical separation and the angle between relevant image keypoints. As a classifier, SVM of various kernels is trained separately. Lastly, the description power of the global feature introduced in this thesis is compared to a well-known feature descriptor, SIFT. The results demonstrate that global shape descriptors that rely on curvature information are competitive and can ultimately lead to a better cropped character recognition.