Towards the development of an autonomous system for driver's drowsiness detection and alertness using a high-fidelity driving simulator

Precise quantification of microsleep (MS) resulting from sleep-disorder and drowsiness rising from insufficient rest in obstructive sleep apnea (OSA) drivers is crucial for preventing accidents. In this work, we propose three methodologies: (i) quantifying MS episodes by correlating driving simulator events with electroencephalography (EEG) patterns; (ii) detecting drowsiness by associating EEG patterns with visual-based scoring through adaptive thresholding for eye-aspect-ratio using OpenCV for face detection and Dlib for eye detection from video recordings; and (iii) optimizing EEG channels and features for their usability in wearable devices. We also propose an autonomous real-time drowsiness detection and alert system for actual bus drivers, aiming to alert drivers through vibrations produced by an electro-pneumatic vibrator located at the back side of the driver's seat and integrated with the existing tire inflation system. Fifty drivers diagnosed with OSA participated in a 50-minute driving simulation wearing six-channel EEG electrodes, while a frontal camera recorded their facial expressions. The first technique identified 970 out-of-road (OOR) events where the wheel and boundary contact lasted ≥ 1 second, and 1020 on-road (OR) events where the wheel and boundary disconnection lasted ≥ 1 second. Applying discrete wavelet transform, theta/alpha ratios were calculated for each event. We classified OOR events with higher theta/alpha ratio compared to neighboring OR episodes as true MS and those with lower ratio as false MS. Comparative analysis focused on frontal brain matched 791 of 970 OOR events with true MS episodes, outperforming other brain regions. Extending our analysis to all channels showed an even higher matching, correlating 923 (95.15%) of 970 OOR events with true MS episodes. We also quantified MS duration, with 95% of total episodes lasting between 1 to 15 seconds, and pioneered a robust correlation (r = 0.8913, p<0.001) between maximum drowsiness level and MS density. The second technique identified 453 drowsy (PERCLOS ≥ 0.3 or CLOSDUR ≥ 2 seconds) and 474 wakeful (PERCLOS < 0.3 and CLOSDUR < 2 seconds) episodes. By applying discrete wavelet transform, we derived ten EEG features and correlated them with visual-based episodes using various comparative criteria. We assessed that the theta-to-alpha ratio exhibited robust mapping (94.7%) with visual-based scoring, followed by the delta-to-alpha ratio (87.2%) and delta-to-theta ratio (86.7%) while considering all channels together. Notably, the frontal area (86.4%) and channel F4 (75.4%) aligned most episodes with theta-to-alpha-ratio, while the frontal, occipital regions, particularly individual channels F4 and O2, displayed superior alignment across multiple features. The third technique normalized ten EEG features for each visual-based episode, then computed seven different thresholding techniques to identify the most consistent method across subjects. Episodes were classified as either drowsy or wakeful based on feature-dependent criteria, evaluating whether their normalized values were above or below a specific threshold. We paired the EEG features into 45 combinations to determine the ideal pair in classifying each episode, in line with visual scoring. Among these, the pairing of PSD alpha and PSD theta in channels F4 and O2 achieved average coverages of 96.1% and 95%, respectively, with corresponding accuracies of 95.4% and 94.7%. These coverages slightly surpassed the results obtained using six channels and a single feature, with an increase of 1.47% for F4 and 0.32% for O2. Lastly, we presented a novel approach towards the development of real-time drowsiness detection and alert system for an actual bus. In conclusion, our work could potentially be employed to assess fitness-to-drive in OSA patients using a driving simulator, reduce hardware and computational demands, and pave the way for an effective real-time drowsiness detection and alert system for actual bus drivers.