Automatic tex summarization system

Automatic document summarization is a process where a computer summarizes a document. In this process, a document is entered into the computer and a summarized document is returned. The summarized document is extremely useful in allowing users to quickly understand the main theme of the whole document and effectively save their searching time.ADS can perform extractive and abstractive summarization tasks. Extractive summarization techniques involve selecting the most important existing sentences, whereas abstractive summarization techniques involve generating novel sentences from given documents. The abstractive summarization approaches require a deeper understanding of the documents. In contrast to the abstractive summarization approaches, extractive summarization approaches are more practical. Most of them represent documents with some structural and semantic sentence features that indicate sentence importance using a sentence score function.In this study, we focus on an extractive text summarization system. In this system we propose a new weighting scheme which can be used in Latent Semantic Analysis based text summarization methods. In order to see the performance of the proposed weighting scheme, we apply the new scheme on four different latent semantic analysis based summarization methods and we show that the proposed weighting factor makes improvements on all of the methods. The performance analysis of algorithms is conducted on the human-generated extractive summary corpora that include four different data sets. The first two data sets are new Turkish data sets prepared for the thesis study. The last two data sets are the most common English data sets that are used in text summarization studies. As a performance measure, for the first three data sets, we use the F-measure score that determines the coverage between the manually and automatically generated summaries. For the last English data set, we supplemented the above metric with the ROUGE evaluation toolkit that is based on Ngram co-occurrence between the manually generated and automatically generated summaries.The system also includes the proposal of a new hybrid system that combines structural and semantic sentence features used for important sentence extraction. The system employs fifteen features one of which is adapted from text categorization to text summarization for the first time. The features are combined by using weights calculated by two approaches. The first approach makes use of a fuzzy analytical hierarchical process which is a manual process that depends on a series of expert judgments based on pairwise comparisons of the features. The second approach makes use of the real and binary coded genetic algorithm for automatically determining the weights of the features. The performance analysis of hybrid system is conducted on the Turkish data sets. As a performance measure, we use the F-measure score that determines the coverage between the manually and automatically generated summaries. Experimental results show that exploiting all features by combining them resulted in a better performance than exploiting each feature individually.Consequently, in this thesis many new approaches about text summarization subject have been proposed and useful results for researches have been produced. It is our wish that this thesis contributes to the studies about text summarization research areas in Turkey and the world.