Railway fastener defects detection and classification based on deep learning techniques

Railway vehicles are widely used in the world and in our country for freight and passenger transportation. Regular maintenance of railway line components is required for safe transportation. There are different types of maintenance to find railway track defects. Among these methods, visual inspection has an important place. Visual inspection is generally done by the staff visually. With the development of technology, it has become important to find vision-based line defects with the railway vehicle. The automatic inspection system implemented with imaging systems and software is faster, safer and less costly than the manual inspection method, where the personel makes visual inspection. The algorithm to be used in the automatic inspection system should have a high accuracy rate and a stable. In the study, fastener defects were detected with a two-stage method from the images taken from the camera system mounted under the train. At the first stage, the position of the fastener was determined according to the rail and sleeper position from the line image with image processing methods. In the second stage, classical feature extraction methods such as LBP and HOG, and KSA-1, MobileNet-v3, ResNet-50 and RegNet-1.6GF models in different Convolutional Network architectures were used for fastener condition detections. Deep learning models gave better results than classical feature extraction methods. In the RegNetY-1.6GF model, which gave the best performance, 98.71% classification accuracy in the test data and F1 scores of 0.992 for an intact fastener, 0.992 for a missing fastener, 0.98 for a damaged fastener, and 0.976 for a rotated fastener were obtained.