The position determination of the objects from images taken by a pair of camera for walking robots

In the mobile machines such as robots or vehicles, it is unavoidable that to have a detection system that give the ability of detection of objects around and prevent the crash of the machine. Especially the ones which have no human control there must also be a desicion making mechanism additional to detection systems. In these kind of systems where there is no human control the importance of the detection system increases. They must see the objects not only in a linear direction but also a wide angle so that they can move without supervision of a human. In this requirement, cameras are good solutions because they have enough angle of vision which is close to a human eye. However, cameras have a 2D vision and it is not possible to get the depth information directly. To get the depth information from a 2D vision lots of studies conducted in recent years. The main principal in getting depth information from 2D views is firstly deceting the edges. By detecting edges it is possible to process very less data and reduce unnecessary information without deforming the structural properties of the image. Therefore it is one of the most important steps to detect objects and get depth information. There are many ways to perform edge detection, however, the majority of the different methods may be grouped by "gradient based edge detection" and "laplacian based edge detection". In the gradient method, the edge detection is done by looking for the minimum and maximum points of the first derivative of the signal coming from the image. In the laplacian method, it is looked for the zero crossing in the second derivative of the function. In the human vision system, the left and right eye see the view from different angles, and these images are somehow processed in the brain which lets us to get the depth information. By using two cameras which are fixed on the same line and have a constant distance between them, it is possible to simulate human vision. The problem is to simulate what the brain does. This thesis aim to solve this problem by developing a source code which can process the images and get the depth information. To achieve this, firstly the images are converted to mathematical model, than the images are put in to the same frame. The distance between the same pixels on different images is detected. Than, by using the stereo vision geometry, the deth information is calculated. In this study, C++ programming language is used to process the images. Also there are some libraries which are specially developed for image processing, in C++. OpenCV is one of the most popular open source library which is developed by Intel Russia. OpenCV is a library which is develop by comprehensive studies and includes optimized codes for image processing and these codes are free and open for everyone to develop new software for image processing. CV and HighGUI are the main components of OpenCV which are widely used. CV has all computer vision algortithms which are used in image processing. HighGUI is used for reading, writing and screening and also to create slide bars to set parameters. To process the images correctly, the cameras have to be calibrated. Practically, any object can be used as a calibration target as long as the coordinates of the target are known in reference to the camera. However in this study chessboard is used which is one of the most popular one. After getting the images from calibrated cameras, rectification process is done. The rectification of the image pairs is putting the corresponding pixels in the same line. More formally, epipolar lines becomes collinear afte rectification. Rectification is commonly performed to improve the speed of the search for stereo correspondences. Given two cameras where the intrinsic parameters are known, the images undistorted, and the image planes have been rectified, the cameras can be calibrated so that the rotation and translation of one camera is known with respect to the other. These relations are found in the rotation matrix and the translation vector. The rotation matrix contains parameters that rotate the left camera so that its image plane is mathematically coplanar with the right camera. The translation vector relates the positioning, or offset, of the left camera with respect to the right camera. The correlation process, in which the depth information is calculated, follows the rectification. In this process, rectified images are used to match the points from one image to the other. In order to determine the distance from the camera, the disparity needs to be found which is the change in location of points in the left image to the right. It follows that there must be an overlap in the two images so that a point in the left image also exist in the right image and a correspondence can be found. An algorithm that can be used to compute correlation and find disparities is a block matching technique where sum of absolute distances (SAD) windows are used to find correspondences. SAD windows are used as a scoring method for each pixel in the image based on its surrounding neighbors. There are three steps to the block matching technique that OpenCV uses; prefiltering, correspondence search, and post filtering. In prefiltering step, the left and right images are normalized by a filter. In the correspondence search step, the matcing points in the image pairs are found. The point in the right image within the search area with the lowest score is considered the best match for the point in the left image. The offset of this point from the original point in the left image is taken as the disparity for that correspondence and from that information the depth is computed. Points with large disparity values represent points that are closer to the camera and smaller disparities represent points that are farther away from the camera. Post filtering is done to remove correspondences that are considered false matches. For this, OpenCV uses a uniqueness ratio as well as a texture threshold. The uniqueness ratio is used to make sure that the value that was calculated for the matched point is not just the closest score, but is an outlier score where it is surrounded by scores that are far from being a match. The texture threshold is set so that noise can be reduced during the matching process, not score that is below the texture threshold is considered. In this study, in order to determine the distance, a laptop and two cameras are used as hardwares. Cameras are connected to laptop by USB ports. The operation of two cameras at the same time and all image processing codes are created in Visual Studio and C++ programming language. The code accepts two images and a calibration data file as arguments. The calibration file was created using the Matlab toolkit during the calibration phase. The application is set up to process the images automatically and then display the distance. A slide bar window was created so that various parameters dealing with the correlation process can be adjusted. With the code developed distance determination is done however the calibration of the camera will be improved to get better result.