Extraction of named entities from Turkish document collections

This thesis aims to develop a model improving Hidden Markov Model (HMM) and Conditional Random Field (CRF), which are two common sequence classifier techniques, for Named Entity Recognition (NER) task on Turkish documents. So, we first examined for the best values of parameters used as input in these models. In HMM, we represented each token with multi features. Next, we used CRF model to determine most effective parameters values that are used as input in this model such as window size, output encoding format and features extracted from tokens. After detailed examination of both HMM and CRF models, we applied a linear-chain CRF model, for NER in Turkish documents. Besides, we proposed 41 different features in four categories: rule based, lexical, dictionary lookup and morphological based features. First, we performed a set of experiments using this feature set on publically available NER datasets. We achieved the best performance with a linear-chain CRF model using [-3, +3] as a window size, BIO encoding as an output encoding format and extended feature set. In terms of F1 measure, we obtained the 91.83 percent, 91.2 and 88.62 for person names, location names and organization names respectively. Furthermore, this thesis also presents METU-NER corpus, which is based on annotation METU corpus for NER. We evaluated our a linear-chain CRF model with the same parameters used in the previous dataset. In terms of F1-measure, we achieved 73.26 percent, 70.12, 63.83, 63.83 and 69.14 for person, location, organization, temporal names and overall, respectively.