Real-time object detection and recognition on FPGAS's by using deep learning

Object detection and recognition are ones of the main tasks in many areas such as autonomous vehicles, robotic, and medical image processing. Recently, deep learning has been used in these areas by many researchers when the data measure is large. In particular, one of the most up-to-date structures of deep learning, Convolutional Neural Networks (CNNs) has achieved great success in this field. Real-time studies related to CNNs are carried out as embedded by using Graphics Processing Unit (GPUs). Although GPUs provide high stability, they need to high power and energy consumption and large computational load problems. In order to overcome this problem, it has started to be applied on the FPGA-Field Programmable Gate Array in the field of binary precision weight and activated CNNs called as Binary Neural Network (BNNs). In this study, object detection and recognition processes were performed using the PYNQ FPGA board including both the ARM processor and the ZYNQ XC7Z020. Firstly, Haar-Cascade classifier was created using the Open Computer Vision Library (OpenCV) installed on the PYNQ board, face and eye detection were performed. Secondly, the identification of objects such as cats, dogs, vehicles, aircraft-like objects was carried out using the BNNs developed in the literature. Real-time recognition was performed with pre-trained BNNs on the object image stored in the memory or on the image taken with the external camera attached to the PYNQ board. The required time for hardware and software and their performances are given by tables and figures. Thirdly, MNIST handwritten and CIFAR object image datasets were used for comparing those of PYNQ with both Central Processing Units (CPU) and Nvidia TK1 and TX1 GPU boards in terms of speed and stability. Finally, real-time object recognition applications have been made with the Movidius USB-GPU externally being plugged into the PYNQ. The obtained results were given with figures. In this thesis, it has been concluded that FPGAs have faster results with lower power consumption if the set of GPUs is used for object detection and recognition. Key words: FPGA, PYNQ, Object Detection, Object Recognition, Deep Neural Networks, Binary Neural Networks.