Product recognition and counting in retail stores using computer vision

Retail business is a very competitive industry and retailers are using every possible means to maintain and grow their businesses. On the other hands, computers are available in almost every retail store, and are usually used to perform traditional tasks, such as sales and financial managements. Moreover, some more sophisticated methods are proposed to assist retailers offering better services to the customers in order to maximize the profits. Some of these methods are used to support the decisions made by the retailers, while other methods are used to monitor the execution of these decisions. The use of computer vision is found to be more cost efficient than other techniques, but it has its own limitation in classification, especially that most of the computer vision techniques convert the image to grayscale before any processing to detect shapes rather than colors. Thus, these techniques may be considered as color-blinded, while many products are different from each other mainly in color. In this study, computer vision is used to monitor the placement and number of products on the shelves of a retail store by combining the Speed Up Robust Features (SURF), which is used to detect the existence of one image into another, and the color average, to improve the classification performance, especially for products of same brands. The results of the conducted experiments show the huge improvement in products recognition when the color average is used alongside with the matching ratio resulted from the SURF method. In experiment (A), the accuracy is measured for products recognition using the SURF method only. The detected products are 95% of the total number of products in the images, 61% of these detected products are classified correctly. Then, the color average is used alongside the matching ratio, in experiment (B), and the number of products detected in this scenario is still 95%, because the detection task is achieved by the SURF algorithm only, and the color average has no role in this task. On the other hand, 88% of the detected product are classified correctly in this scenario, which means that the use of color average improved the results by 27%. In experiment (C), three model images per product are used for training, instead of one model image per product in the previous scenarios, to improve the recognition capabilities of the SURF method. The detected products in this scenario are 99% of the total number of products in the tested images, and 97% of these products are classified correctly into their corresponding classes. The results of these experiments show the huge improvement in products classification when the color average is used with the ratio of the matched interest points to the total number of interest points in the model image. This improvement is according to the fact that the SURF detects interest points regardless of the colors in that image, while the main difference among products of the same brand is usually the colors of these products. Moreover, the increment of model images per product in the training dataset results a better detection, as well as better classification, especially when model images taken for each product are in different lightings and view angles.