Effect of token selection on Huffman coding

In this study, the effect and efficiency of token selection is investigated on the Huffman compression algorithm, one of the statistical data compression methods. To this end, compression gains for different types of tokens identified using regular expressions to produce Huffman tree is calculated and compression performance is compared. The study consists of five main chapters. In the introductory chapter, it is mentioned that the definition of data compression and classification of the data compression methods. In the second chapter, statistical data compression, one of the data compression methods is examined and basic concepts in information theory are explained. In the third chapter of the study, to describe used token type, it is introduced n-gram, Turkish syllabification algorithm and regular expression concept. Also in the fourth chapter, as well as n-gram, syllable and regular expression, Huffman trees with tokens created with collocation of their is generated and compression processing is performed. Compression processing is tested on seven different documents and the results of each document that is used for all tokens type is obtained. In the last chapter of the study, the results obtained is discussed.