Defect detection and classification on metallic surfaces using deep learning methods

In recent years, the need for automatic defect detection systems has been increasing in the production phase. Identifying and locating product defects is an important and necessary quality control process. Detecting the type of failure and the defective area in a short time is also very important in terms of quality control performance. Surface defect detection is common in many industrial applications, such as textile, metal and glass flaw detection. There are various and complex types of defects (patches, inclusions, scratches, pitted surface, rolled scale, etc.) on metal surfaces. Fastness and high accuracy in real-time metallic defect detection systems have a positive effect on the production stage. Labeling data manually is a time-consuming process. Therefore, using algorithms can accelerate and optimize this process. This also reduces human error in the labeling process. Algorithms can automatically label data based on predefined characteristics, providing a significant advantage over manual labeling by humans, resulting in faster and more accurate results, especially on large data sets. Applications developed with this method enable data scientists to analyze data faster and obtain more accurate results. Today, human inspectors are still used to detect defects as a traditional method in the manufacturing process. Computer vision techniques are frequently used in production systems for quality control to increase production speed, reduce error rates and prevent human errors such as fatigue and distraction. Morever, the labeling process carried out before the training is still done manually. Defect regions must be individually labeled when classifying with Faster R-CNN before the training process. Automating this process is vital in industrial control systems, as it is very time-consuming to label the dataset one by one. This study aims to automate the labeling process and save time by synthesizing and using SFS and Deep Learning methods. In order to achieve successful performance in object recognition and classification applications with Faster R-CNN, a large number of labeled data is needed in the training process. For example, the NEU and KolektorSDD2 datasets used in this study contain 1800 and 3335 images, respectively. While manually labeling the images takes a long time, automating it with SFS yielded considerably better outcomes in a lot less period. Traditional methods for defect detection on surfaces rely on image processing or machine learning techniques, but they can detect different defects with low noise and strong contrast at certain scales or under certain lighting conditions. Today, computer vision and deep learning approaches have an important place in industrial control systems. Computer vision technology is essential for fast and accurate control of products on the production line. The importance of the concept of computer vision is understandable when considering the problems of classical methods. Metallic defect detection is a challenging problem as metal surfaces are easily affected by environmental factors such as lighting and light reflection. Since traditional detection algorithms are inefficient in complex problems, we propose a novel method to detect and classify metal surface defects, such as cracks, scratches, inclusion, etc. The type and location of defects were detected by the Faster Regional Convolutional Neural Network (Faster R-CNN), combined with the Shape From Shading (SFS) method, which can extract surface characteristics. The Northeastern University (NEU) surface defect database was used for defective samples. The proposed algorithm has also been tested on an unlabeled dataset (KolektorSDD2/KSDD2) to show labeling performance. The results on both labeled and unlabeled datasets have demonstrated state-of-the-art performance in automatic defect detection, classification, and labeling. The proposed method has satisfactory results for the detection of defects on the metal surface, and the mean average precision is 0.83. The average precision of crazing, pitted surface, patches, scratches, inclusion, and rolled-in scale are 0.98, 0.81, 0.90, 0.79, 0.88, and 0.62 respectively.