Analysis of histopathological images via machine learning methods

In recent years, deaths caused by cancer and diseases are more prominent than deaths caused by other diseases. Early diagnosis of cancer is very crucial for the treatment of this disease. With the development of imaging devices, it has become possible to treat the disease, follow-up and treat with the help of Computer Aided Diagnostic (CAD) systems. Particularly with high resolution scanners, it is possible to automatically detect changes in tissues and organs by CAD systems. This thesis consists of two main sections, namely the segmentation and classification of high resolution histopathological images. Superpixels based and deep learning based semantic segmentation algorithms are used for the segmentation of cellular structures. The performance of SLIC, SLIC-DBSCAN, ERS and TPRS superpixels segmentation algorithms, which are frequently used in computer vision, have been tried to be obtained in the segmentation of cellular structures. For segmentation purpose, a new segmentation algorithm is proposed in order to segment the cellular structures in the histopathological images by combining the SLIC superpixels segmentation algorithm and clustering-based algorithms such as k-means and fuzzy c-means. The performance of the proposed method was compared with other well known methods used in the literature. In addition, deep learning-based semantic segmentation (SEGNET) method, which is a very successful method in object detection, tracking of moving objects, segmentation of objects in outdoor, is used in the segmentation of cellular structures. In the "classification of histopathological images" section, the detection of mitotic cells and tumor regions located in the lymph nodes was carried out by using the convolutional neural network method which is very popular in recent years. In the detection of mitotic cells, performance analysis is performed by comparing the conventional shape, color, texture and statistical based feature extraction methods with the deep learning based methods. When the results are examined, it is observed that the ESA model perform better classification accuracy than the conventional methods. In addition, it is observed that the performance can be increased by using sampling based classification methods which are robust against unbalanced data such as RusBoost. Finally, in cooperation with Istanbul Medipol University Hospital, Istanbul Technical University and Yildiz Technical University, one of the largest whole slide image data set in the literature regarding the grading of cervical cancer precursor lesions has been created and classified.