Otonom sürüş için sensör birleştirme yaklaşımı

In this thesis, a sensor fusion approach between camera and lidar sensors is proposed to overcome shortcomings of single sensor setups. For camera sensor, a deep learning model, a multitask network is proposed to localize dynamic and static traffic objects and segment drivable area and lane lines. Dynamic objects include cars, motorcycles, bicycles, buses, trucks and pedestrians, static traffic objects include color classified traffic lights and traffic signs. This proposed multi-network achieves multiple tasks by itself faster and in real-time as normally there would be separate network models for each of these tasks working on parallel. The multi-network proposed is trained and tested with Berkeley Deep Drive 100K dataset. Evaluation results show that the proposed method is the fastest multi-network on the dataset with 47.62 FPS. Around multi-networks, proposed model has the second place on drivable area segmentation and lane line detection. Dynamic object localization performance of the network is state-of-the-art with 40% performance increase compared to other models. For Lidar sensor, 3D object detection and Lidar ground plane detection are achieved by already proposed methods. Finally, the detections from two sensors are fused by proposed fusion algorithms and results are evaluated with KITTI dataset. The proposed fusion approach exceeds the performance of Lidar-only methods up to 9.87% category wise. Comparison with two different fusion approaches also show our superior performance. Proposed sensor fusion methods can work in real-time by taking only 36 milliseconds, nearly 33 frames per second.