Development of a turkish mobile assistant software using natural language processing techniques

Since the interaction with mobile devices increased enormously in recent years, user friendly interfaces facilitating human-machine interaction became very important. Since natural language is the most natural way of communication for humans, mobile assistant applications became very popular. Especially after SIRI gained popularity, several companies developed similar applications and released to the application markets. Nevertheless most of these applications supports English and provide services mostly for English-speaking countries. Unfortunately, the challenges caused by the typological differences of some natural languages also appears in mobile applications and the generated systems which are mostly on English, could not be adapted easily to other languages such as Turkish. In the current state, there is no application for Turkish that uses advanced natural language processing (NLP) solutions, supports many services in a flexible structure and has a high success rate on understanding users' request except SIRI which started to support Turkish language after IOS 8.3 update in April 2015. Even though SIRI began to support Turkish, its success rate and supported service count is very low compared to its English version. The goal of this thesis is to illustrate the impact of using natural language solutions on mobile assistant applications and develop a new application that will support Turkish natural language input with high success rate. The application developed performs operations on mobile device (e.g. calling someone) and retrieve and show information (e.g. weather forecast information). A virtual assistant is an application that contains almost all components of artificial intelligence. Voice recognition, voice generation, natural language processing components to understand the voice semantically, and even usage of 3 dimensional holograms to simulate humans (image recognition or generation) can be considered in this aspect. Mobile assistants are applications that interacts with user over voice processing and understands user's intention and accomplish his/her request. In the last 15 years successful voice recognition, voice generation and natural language processing components have been developed for Turkish. The researches in these fields are still continuing actively. However the components have not been integrated in a high level application. Within this thesis, domain adaptation is applied to these researches (speech to text, morphological analysis, dependency parser, named entity recognizer, text normalizer) in personal mobile assistant domain and combined in an application. It is observed that Turkish mobile assistant applications which there is only few, developed in a rule based approach and does not use or uses very basic natural language processing components. Therefore, their success rate on understanding and executing user's query is very low. For example for a query such as "Ankara'da hava bugün nasıl? (How is the weather today in Ankara)", such applications return the weather forecast of the current location (e.g. "Istanbul") instead of the requested location ("Ankara"). Implementing Turkish NLP researches to the software would increase the success of these applications. This work is the first academic study focusing on the language related problem of a Turkish Mobile Assistant and shows the state-of-the art in this field. It is expected that the application and study in this thesis will become a pioneer in its field of practice. The application developed is able to understand and fulfill voice queries on following domains. Sending message (Gökhan'a yarın akşam gelemeyeceğimi mesaj at) (Send message to Gökhan that I will not come tomorrow evening), Sending e-mail (Gökhan'a yarın akşam gelemeyeceğimi mail at) (Send email to Gökhan that I will not come tomorrow evening), Calling a number or contact (Gökhan'ı ara) (Call Gökhan), Launching applications on the device (Facebook'u başlat) (Launch Facebook), Search something on the Internet (İstanbul'u Google'da ara) (Search Istanbul on Google), Retrieve weather information (Bugün hava nasıl olacak?) (How will be the weather today), Map Direction Google Maps (Beşiktaş'a nasıl giderim?) (How can I go to Besiktas), Currency exchange (10 dolar kaç lira) (How many liras is 10 dollars), Traffic information (Şu an İstanbul'da trafik nasıl?) (How is traffic in Istanbul). Developed system consists of 2 modules which interacts with each other using defined JSON protocol. Mobile application is responsible for 1. capturing the spoken request, 2. converting it to written text 3. transferring the user requests to the server side and 4. displaying and/or performing the requested operation according to the result received from the server. Server service is responsible for processing the user's query and understand its true intention. It also collects information from third party web services if needed, and finally, compiles the results and sends it back to the client. Google's speech recognition API for Android is used in mobile application in order to capture the user spoken queries and convert them into written messages. The speech recognizer returns more than one text suggestions for a given entry and each suggestion has its own confidence level. If any of the produced suggestions has more than this threshold value, it is the assumed to be the correct suggestion. Otherwise, the application presents a list of text suggestions to the user in. After this stage, the selected query text is sent to the server along with the user's context, which includes the user id, current location and time. The returned answer from the server is then either executed on the phone or displayed to the user. On the server-side incoming user query is first processed with NLP tools, and then the query is mapped to one of the supported operations through a hybrid approach of rule-based and statistical classification. Different algorithms have been tested in statistical classification step and best result is obtained with support vector machine (SVM) algorithm with 95.80% classification accuracy. Total classification success of the hybrid approach is measured as %98.30. After this stage, the required parameters for each operation are extracted from the query using the assigned analysis by the automatic natural language processing modules. Whole system success is measured as 70.79%.