Named entity recognition in Turkish

For decades humankind dreams on interacting with machines via their natural spoken languages. There are several interesting research project related to this topic. Speaking with machines or computer systems, making them understand natural languages, solving the problems for humans are the main theme of many science fiction books and movies. Making the human-machine interaction with natural languages, automated language translations, semantic analysis of textual data and many other natural language centric operations are tasks related to Natural Language Processing research area. When the topic comes to extracting information from textual content recognition of named entities from the text is a very important step for the task. Recognition of person, organization or location information, numerical values, date and time expressions from the given natural language text is called Named Entity Recognition (NER). In many NER task recognition of person, organization or location information is called ENAMEX, recognition of numerical values, money and percentage expressions called NUMEX and recognition of date and time expressions called TIMEX entity types. Besides NER task is not limited with ENAMEX, NUMEX and TIMEX entity types. NER studies begin in 1990's and increase popularity after 1996. Besides the first known study about NER in Turkish Language made in 1999 and popularity of NER systems for Turkish Language increase after 2009. Languages like Turkish or Finnish are agglutinative languages. They have complex morphological and affix based structures. In contrast to Turkish or Finnish, English has more simple language and morphological structure. This language structure of English easier to solve many Natural Language Processing task including NER in English. There are many research about NER task for English. It could be accepted that NER is a solved task for English. On the other hand languages like Turkish or Finnish has a more complex morphological architecture then English, besides researches are still limited in Turkish. This makes NER task still a hot topic for Turkish Language. When researches about NER tasks are analyzed the first studies about NER are task recommends rule based systems. The researchers proposed that it's easier to design rule based systems when training data about the task is limited. On the other hand creating hand crafted rules to detect and tag named entities is a complex task. Besides rule based systems also suffers for context dependency. When changing the context of the system, performance of the system decreases gradually. Recent studies in NER task focus on statistical and machine learning methods. There are featured studies using especially Conditional Random Fields (CRF) in Turkish NER tasks. On the other hand in many other languages NER systems based on artificial neural networks and deep learning creates outstanding performance in NER task. Artificial Neural Networks (ANN) is a machine learning methodology which models human neural system. In human neural system neuron cells are the basic architectural structure. In artificial neural networks a neuron like structure called perceptron is the basic architectural structure which makes computational operations. Theoretical ANN studies begun in 1960's however the complex computational operations in ANN's learning step requires advanced computational hardware. This prevents usage of ANN based system until recent decades. Developments in hardware systems made ANN based systems accessible, moreover more complex architectural neural network implementations became possible in recent years. There are several different ANN architectures focus on different machine learning problems. Basically ANN's are used with Feed-Forward Neural Network structure. In a Feed Forward Neural Network percentrons connected parallelly and serially with each other. Each percentrons in the same level called a layer and in Feed Forward Neural Networks loops are forbidden between layer. There is no memory unit in Feed Forward Neural Networks, no state or data can be transferred between usage sessions of the network. Because of the simple architecture, implementation and learning phase of a Feed Forward Network is quite simple. On the other hand there is another neural network type called Recurrent Neural Network (RNN). In RNN's output of the network transferred as an additional parameter to the input at the next usage of the network. Also loops are acceptable between inner layers of the neural network. Thus Network can share and transfer state or data between usage sessions of the network. So RNN's perform better results compared with Feed Forward Neural Networks when processing a sequentially related data. However RNN models have computational complexity in both learning and operation phase. This made RNN implementations to request advanced computational hardware systems. Furthermore in theoretically RNN's can handle and transfer state between usage of networks so they can handle sequentially related data better compared to Feed Forward Neural Networks. However there is a problem called Vanishing Gradients explains that transferring state between short usage sessions of network is acceptable but in long term state transfer between sessions are limited. To solve the Vanishing Gradient problem it is suggested to use a special RNN architecture called Long Short-Term Mermory (LSTM). LSTM contains a seperate memory network unit that responsible to store, update and transfer state between sessions. Thus LSTM can handle transfer an important state information between long running sessions. Transferring state information in classic RNN or LSTM system is only forward only. In other words RNN or LSTM systems cannot solve dependencies in backward direction in a sequential data because they cannot transfer the state information to previous sessions. Bidirectional networks used in order to solve this problem. Bidirectional LSTM (BLSTM) consist of 2 LSTM layers which one operates in forward direction and other layer operates backward direction. The output of the forward and backward layers generally passed over a Feed Forward Neural Network to find the output. BLSTM structures are performs successful results when sequential data is related both in forward and backward direction. Last, there is another RNN model that uses stacked BLSTM networks called Deep BLSTM (DBLSTM). In DBLSTM architecture BLSTM layers are serially connected to each other. In recent studies its reported that DBLSTM architecture produces outstanding performance in contrast to single layer BLSTM networks especially in Natural Language Processing tasks. Therefore in this thesis a Named Entity Recognition Systems for Turkish Language based on LSTM, BLSTM and DBLSTM architectures are designed and implemented. Both design of the systems and the implementation specifications are reported in detail. Besides in order to measure and compare the performance of the implemented systems a well known Turkish NER Dataset used in the experiment section. As a result we reach an F1 score of %93.69 in the best system. As far as we observe the best result in literature has %93.59 F1 score. Our best system result reaches and smoothly improves the best result in literature.