Product search engine using product name recognition and sentiment analysis

In this study, a novel product search engine system which consists of a focused crawler, a record linkage system and a sentiment analyzer is proposed. We develop an original focused web crawler for E-commerce sites, and the challenges and our proposed solutions are presented in detail. A sentiment analyzer is developed to classify E-commerce product comments into polarities as negative or positive. A novel record linkage system for E-commerce products is proposed to recognize the same product names collected from different E-commerce sites. The record linkage system is based on a modified dynamic/incremental Hierarchical Agglomerative Clustering algorithm which employs our proposed product code matching system to reduce number of product name comparisons during clustering. In addition to these systems, a search system and a user interface are developed for the product search engine. In this thesis, we present a full scale product search engine that obtains %472 performance boosts in the crawler, 91.08% accuracy in the sentiment analysis, 96.25% F-measure in the record linkage, and 100% precision in most related products search. The proposed system achieves to provide better user experience than the existing systems.