Development of a new deep learning model for video detection of unsafe behaviors in industrial environments

Unsafe behaviour can occur as a result of negligence and lack of due caution in production plants, businesses, factories and other places where people are present. In such areas, unsafe hazardous behaviour is a major cause of death or injury, including many accidents. Despite traditional occupational safety practices and regular safety inspections of workplaces, many accidents occur as a result of violations of occupational health and safety protocols. These accidents show that existing safety control procedures are not adequately implemented in workplaces, and that the dynamics and challenges of the work environment are not adequately addressed. Although solutions to prevent accidents and losses in hazardous environments have reduced the number of incidents over the years, they cannot be completely eliminated due to human behaviour. In addition, such production environments are highly complex, have lighting problems and are extremely dynamic. Although various systems exist to control hazards in work environments, it is clear that there are very few real-time approaches. In particular, although there are many computer-aided automated solutions, the training and detection process of these systems has low accuracy, is very costly and time consuming. On the other hand, the use of personal protective equipment (PPE), which is one of the most important elements of occupational health and safety, is of great importance in industrial workplaces. In this thesis, the appropriate use of PPEs was first determined by identifying multiple classes using the YOLO (You Only Look Once) learning algorithm. At this stage, seven classes of PPEs were identified and a unique dataset was created for these classes. In the experimental studies carried out to detect PPEs, the mean average precision (mAP) value achieved using the YOLO architecture was 91.18%. Furthermore, for the other metrics, precision, recall, F1 score, intersection over union (IoU) and average loss, the results obtained were 0.89, 0.91, 0.90, 70.35 and 1.1147 respectively. In the second phase of the thesis, Unsafe-Net (Önal & Dandıl, 2024a), a hybrid computer vision approach supported by deep learning models, was developed for real-time classification of unsafe movements in workplaces. For the Unsafe-Net infrastructure, a dataset was created by collecting 39 days of video footage from a factory. This dataset was published in the journal Data in Brief in 2024 and made available for available (Önal & Dandıl, 2024b). Using the database created specifically for the study, YOLOv4 and Convolutional Long Short-Term Memory (ConvLSTM) deep learning architectures were combined in object recognition and video interpretation to achieve fast and accurate results. In the experimental studies at this stage of the thesis, the classification accuracy of unsafe behaviours in workplaces using the proposed Unsafe-Net architecture was achieved as 95.81% and the average time for action recognition from videos was calculated as 0.14 seconds. In addition, thanks to the YOLO algorithm used in the infrastructure of the Unsafe-Net architecture, the average video duration was reduced to 1.87 seconds, thus increasing the real-time detection speed. The developed system has been installed in a factory environment operating in Eskisehir, and when unsafe behaviour occurs, employees are immediately alerted by the system both audibly and visually in real time. In addition, with the installation of the system in the factory environment, employees were monitored for a period of time and it was observed that the recurrence rate of unsafe behaviour decreased by approximately 75% in a short period of time.