Vehicle driver state assessment using multi sensory image modalities

In this thesis driver head movements are investigated, which is directly related with state of the art active driving safety. Here, driver face images are assessed using multisensory image modalities. It is well known that we can understand driver head movements to get a drowsiness or distraction measure as he/she is on a steeringwheel. Mean shift method is preferred in this study to track the head movements. In order to identify efficiency of this method as compared to asymmetric kernel mean-shift, whose research progress is still ongoing, their performance comparisons are conducted.Driver head database has been built using visual and infrared cameras as preparation of experiments. To sense driver head movements in the database VHD method, video histogram differences, is applied. Then, histograms of motion vectors are drawn to get comparison between optical flows of the images taken by visual and infrared cameras. The histograms of the sequence of head movements are also considered as clues to get head direction and plane of driver. To analyze the driver head movements we investigate performance differences between Bayesian and ARMA modelings using the sequences consisted of visual and infrared images database.In the course of driving, driver and the objects behind him can momentarily be exposed to different light conditions. As a matter of the fact that stereo vision is suggested to preprocess the driver image with the aim of eliminating possible bacground complexity caused by fluctuant lighting conditions. In the progress of the thesis, respective experiments have been carried out to show how to eliminate the cluttered bacground using depth map. Furthermore Curvelets, which has recently emerged in research studies, are also been implemented to carry out experiments using multisensory images which are acquired under different illuminated environments. The OSU thermal image databases, and visual and infrared driver head images which have been obtained in real commuting environment are utilized in experiments. Since cost is one of the principle concerns as competitiveness for a car manufacturer, web-cameras have been implemented and discussed in terms of performance and cost driven.As one of the outcomes of this study, visual and infrared images are fused to get the better system performance on multisensory image models, and the results are evaluated in accordance with the aim of the research.