Text Mining Techniques and an Application on Natural Language Processing by Using R

In our current society, technology is advancing at a very high pace, and these new inventions also generates large amount of data. Data is now increasing at an exponential rate, and this alarming growth rate has led to difficulty in getting and retrieving specific information from the web. Automatic summarization systems can help to resolve this information overload problem in an effective way. It easily identifies the important points from a document to produce a concise summary. Thus, the thesis investigates the extractive-based approach in generation of a summary from single documents/texts. In the study, an extractive-based summarization framework (EBSF) was designed, also, an extractive-based text summarization system has been developed, evaluated and its workflow described. The framework implements several techniques and the summarization system generates extractive summaries from news articles using an extractive-based summarization technique which is based on the TextRank algorithm. Results from the various program testing shows that the summaries generated using our extractive-based summarization system offers an excellent tradeoff between time/length and accuracy. In this study, the summaries from the designed summarizing system, tends to be concise and contain less extraneous material.