Theses supervised by Hasan Demirel
11 theses · Eastern Mediterranean University
3D Tumor Segmentation in MR images using Tumor Characteristics
3D segmentation is an important pre-processing phase that dictates the performance of the possible post processing applications in 3D biomedical image analysis. 3D reconstruction of the segmented tumor from 2D MRI plays an essential role in 3D encapsulation of the segmented tumor. Fuzzy C-mean clustering is widely used in 2D segmentation of the brain tumor. In this thesis 2D tumor projections in each slice of the MRI are segmented using FCM. 2D slices extracted through FCM are used to generate the 3D reconstruction of the tumor. However, the possible low number of slices limits the quality of such 3D reconstruction. This thesis proposes to minimize this problem by generating predicted slices (P) between intra slices (I) and generating bi-predictive slices (B) between predictive slices and intra slices by using bilinear interpolation. 3D reconstruction of the brain tumor is performed by using the real slices and the predicted slices. Limitations of the FCM clustering in deciding which cluster is to be segmented is overcome by employing 2D/3D characteristics such as intensity, circularity and 3D Compactness in T2 -weighted MR Images for reliable segmentation of the 3D brain tumors. The proposed approach leads to detect the 3D tumor automatically with improved accuracy compared to alternative methods in the literature. Keywords: 3D Segmentation; MRI; FCM clustering; Labeling; Thresholding; 3D reconstruction; 3D compactness; Circularity; Tumor; Brain.
3D Face Recognition using Hyperspectral Images
Face is one of the most common biometric modalities which is used for identification. In this context, face recognition has gained an important role in biometric applications based on identification systems during the last few decades. Since there is no physical interaction required during recognition or identification, it's easy to deploy and implement. In face recognition, a face is categorized as known or unknown by comparing a face with all the faces in a database. Due to inherent distinct features, human face analysis is one of the most effective methods of identifying individuals. Nowadays, utilizing hyperspectral images in face recognition is one of the most important research topics in biometrics, since they contain additional significant spectral information compared to 2D images which have only information in spatial dimensions (texture and structure). A hyperspectral image is a data cube containing two spatial and one spectral dimension. Hyperspectral image samples are captured by a hyperspectral camera which operates at multiple narrow bands within the visible spectrum and neighboring near-infrared spectra. Hyperspectral imaging provides new prospects for improving face recognition accuracy since they contain information in both space and spectral axes. Hence significant information for each person regarding the skin based on reflected, absorbed and released electromagnetic energy at different wavelengths can be extracted. Additional spectral information which is not embedded in traditional grey/color facial images provides an opportunity to improve the recognition accuracy. Hyperspectral imaging employs spatial and spectral relationship simultaneously, which improves segmentation and classification in the respective applications. Difficulties encountered in visible light-based face recognition systems, such as the variance in orientation, illumination or expressions can be minimized by employing hyperspectral imaging. Besides these opportunities, hyperspectral images pose some challenges such as low signal to noise ratios, high dimensionality, and data acquisition needs expensive cameras with multiple sampling in visible and nearinfrared spectra. Despite mentioned challenges, hyperspectral images contain more independent and significant information obtained from different sub-bands than 2D images. Hence, hyperspectral images represented in 3D-cubes are by far more capable in classification processes, which is also ideal for spoofing attacks. In this thesis, we propose novel methods for feature extraction for facial hyperspectral image recognition. The main goal of the thesis is to improve the recognition accuracy of hyperspectral face images. In the first method, three different approaches are proposed employing 3D discrete wavelet transform (3D-DWT) to extract features from the subbands generated by discrete wavelet decomposition. Three approaches include 3D-subband energy (3D-SE), 3D-subband overlapping cube (3D-SOC) and 3D-global energy (3D-GE), which extract different feature vector for each approach containing the energy values calculated from different wavelet sub-bands at different levels of decomposition. Feature vectors generated by three different approaches go through a classifier to complete the face recognition task. In the second proposed method, fusion of spectral information into a single 2D image is achieved by band-specific signal to noise ratio (SNR) based weighting. The fusion method assigns weights based on the calculated band-specific SNR values, weighted sum of the bands generate a single 2D face image. Hence, each pixel along spectral axis is fused to a single pixel resulting a 2D output face image for each 3D hyperspectral face cube. In the third method, in order to fuse spectral bands in hyperspectral face cubes, we apply discrete wavelet transform (DWT) to each pixel along the spectral axis consecutively until the spectral vector for each pixel is decimated to a single pixel transforming the 3D input spectral face image cube into a 2D output image. 2D output images obtained by the second and third methods are processed using principal component analysis method and face recognition is performed with the help of a classifier. The experimental results reveal that recognition accuracy of all proposed methods by using standard hyperspectral databases outperform alternative hyperspectral face recognition of the state-of-the-art methods. Keywords: hyperspectral face image, face recognition, discrete wavelet transform, feature extraction, classification, signal to noise ratio.
Dynamic 3D Facial Expression Recognition
In this study, dynamic 3D facial expression recognition is addressed by proposing novel landmark-based and appearance-based approaches. As a preliminary work, a set of geometric landmark-based features are extracted from 3D images, followed by sequential forward feature selection (SFFS) and a two-layered support vector machine (SVM), fuzzy SVM classifier to recognize six basic expressions. Experiments conducted on BU-3DFE data set proved that the proposed algorithm outperforms the conventional methods advocating the effectiveness of geometric landmark-based methods. In the second phase, a novel method using time series analysis of landmark-based geometric deformations is proposed for dynamic 3D facial expression recognition. After head pose correction and normalization, a set of multimodal time series are constructed from the local temporal deformations by applying a sliding window averaging on a comprehensive set of geometric landmark-based deformations (point, distance and angle). This stage is interlocked with facial action unit analysis to identify the key points from facial landmarks. Then, neighborhood component analysis feature selection (NCFS) is utilized to discard redundant features. Finally, adaptive cost dynamic time warping (AC-DTW) is applied to classify six prototypic expressions. Experiments on BU-4DFE data set confirmed the effectiveness of the proposed algorithm. In the third phase, an appearance-based dynamic 3D facial expression recognition is proposed using low-rank sparse codes and a novel spatiotemporal region of interest (ROIs) pooling. 12 ROIs are defined using automatically detected and tracked landmarks in by applying a multi-point tracker. LBP-TOP feature descriptors are extracted from cuboids inside spatiotemporal regions of interests in both texture and depth sequences and are fused to form the feature matrix. Sparse codes are obtained using low-rank sparse coding. Finally, hidden-state conditional random fields are employed to classify six basic expressions. Experimental results on BU-4DFE data set verified that proposed method improves the accuracy of dynamic facial expression recognition in comparison to previously proposed approaches. Keywords: Dynamic 3D facial expression recognition; Spatiotemporal analysis; Geometric landmark-based deformations; Time series analysis; Dynamic time warping; Facial landmark detection; Landmark tracking; Sparse Code; Region of interest.
Improved PCA based Face Recognition using Feature based Classifier Ensemble
Automatic face recognition has been a challenging problem in the field of image processing and has received a great attention over the last few decades because of its applications in different cases. Most of the face recognition systems employ single type of data, such as faces, to classify the unknown subject among many trained subjects. Multimodal systems are also available to improve the recognition performance by combining different types of data such as image and speech for the recognition of the subjects. In this thesis, an alternative approach is used where the given face data is used to automatically generate multiple sub feature data sets such as eyes, nose and mouth. Feature extraction is automatically performed by using rough features regions extracted from Viola-Jones face detector followed by Harris corner detector and Hough Transform for refinement. Automatically generated feature sets are used to train separate classifiers which would recognize a person from its respective feature. Given separate feature classifiers, standard data fusion techniques are used in the form of classifier assembling to improve the performance of the face recognition system. 10-Fold cross validation methodology is used to train and test the performance of the respective classifiers, where nine fold is used for training and one fold is used for testing. Principal component analysis (PCA) is employed as a data dimensionality reduction method in each classifier. Five different classifiers for right and left eyes, nose, mouth and face data sets are developed using PCA. The classifiers, of five different features are merged by different data fusion techniques such as Minimum Distance, Majority Voting, Maximum Probability, Sum and Product Rule. Overall, the proposed algorithm using the Minimum Distance improves the accuracy of state-of the art performance from 97.00% to 99.25% using ORL face database. Keywords: Face Recognition, Face Detection, Cross Validation, Viola-Jones Detection, Feature Extraction, Minimum Distance, Data Fusion, Classifier Ensemble.
Parkinson’s Disease Detection Using Structural MRI
Parkinson’s Disease (PD) is the second most encountered neurodegenerative disorder, second only to Alzheimer’s Disease (AD), and the most common movement disorder affecting 1% of people over the age of 60. PD is characterized by progressive loss of muscle control that causes trembling of the limbs and head at rest position, rigidity, slowness, impaired balance, and later on a shuffling gait. As the disease is progressed, difficulties in walking, talking, and completing basic tasks might occur. The causes of PD are unknown, yet it is believed that both the environmental and genetic factors might lead to PD. High-quality images obtained using neuroimaging methods could give beneficial support to the clinicians for evaluating the treatments. Threedimensional magnetic resonance imaging (3D-MRI) has been effectively utilized in the detection of progressive neurodegenerative diseases including PD. Therefore, using neuroimaging techniques with Computer-Aided Diagnosis (CAD) has gained increasing attention in the early and accurate diagnosis of PD. In this thesis, the extensive reviews on the studies of PD detection using MRI data and CAD methods since 2008 are studied. Furthermore, the affected brain regions owing to PD are obtained by using the 3D Volume of Interests (VOIs) and the captured affected brain regions are compared with the regions reported in the-state-of-the-art studies in the last decade. The obtained affected brain regions might shed light on the existing literature on PD diagnosis. In order to build an automatic method for PD detection, machine learning algorithms are applied to the processed neuroimaging data. It is obvious that pre-processing of 3D-MRI scans plays an important role for post-processing. In this thesis, the preprocessing of 3D-MRI data has been performed by using a Voxel-Based Morphometry (VBM) technique which evaluates the whole brain morphology with voxel-by-voxel comparisons. In VBM, some parameters such as covariates need to be defined to build a model for Gray Matter (GM) and White Matter (WM) volumes of Structural MRI (sMRI) datasets. In this thesis, the effects of using different covariates (i.e. total intracranial volume, age, sex and combination of them) on the classification of PD groups from Healthy Controls (HCs) have been studied. Additionally, in order to determine the 3D VOIs, the significant local alterations in GM and WM volumes of PD groups and HCs, a hypothesis either f-contrast or t-contrast need to be defined. In this thesis, the effects of two different hypotheses on PD detection have been investigated. Furthermore, a feature-level fusion technique in which the 3D GM and WM VOIs are combined considering the effects of both GM and WM volumes in PD diagnosis. The voxels extracted from 3D GM, WM, and the combination of GM and WM VOIs are considered as raw features. Even though the feature extraction decreases the number of features in raw data, using an automatic feature selection method from high dimensional feature space is an asset in PD classification. In this thesis, to select the most discriminative attributes from high-dimensional data, all raw features are ranked by using various feature ranking methods such as minimum redundancy maximum relevance, Relief-F, unsupervised feature selection for multi-cluster data, Laplacian score, regularized discriminative feature selection for unsupervised learning, correlation- based feature selection, and feature selection and kernel learning for local learning-based clustering. In order to select the optimal number of top-ranked discriminative features, a Fisher Criterion (FC) is calculated for different sizes of feature vectors and the optimal number of top-ranked features is selected when the vector size maximizes the FC. In order to classify the PD and HC, five different classification algorithms, namely k- nearest neighbor, naive Bayes, ensemble bagged trees, ensemble subspace discriminant, and support vector machines are used. Moreover, a decision fusion technique which combines the binary outputs of all five classifiers by using a majority voting method is investigated to achieve higher performance in PD diagnosis. The experimental results indicate that the proposed methods are reliable approaches that are highly competitive with the state-of-the-art methods in PD classification. Keywords: Parkinson’s disease, structural MRI, covariates, f-contrast, voxel-based morphometry.
Person-Dependent and Person-Independent Analysis of Emotion Recognition using Facial Expressions
Facial emotion recognition is one of the prospective fields which can have various applications in many different areas. However, there is a huge difference between a personalized and non-personalized emotion recognition. In facial expression analysis, learning process starts with person’s facial structure. A person-dependent system will receive person specific features during training which is advantageous compared to a person-independent system. Hence, with the addition of ethnicity, cultural background or gender differences, gathering results on non-personalized system of emotion recognition becomes a challenge. In this thesis, models for person-dependent and person-independent emotion recognition are proposed. Experiments are carried out using SAVEE and RML facial video databases. Initially, frames and corresponding landmark features are extracted from the videos. K-means clustering algorithm is applied to the extracted landmark features in order to get the k most significant frames. After representing each video sequence with k keyframes, Support Vector Machine classifier is used for the training and testing of the proposed system. Experimental results show that recognition performance of person-dependent model is higher than person-independent model. Keywords: Machine Learning; Image Analysis; Emotion Recognition; Facial Emotion Recognition; Support Vector Machine
Structural MRI - based classification of alzheimer's disease
ABSTRACT Alzheimer’s disease (AD), an irreversible neurodegenerative dementia, occurs most frequently in older adults which gradually destroys regions of the brain that are responsible for memory, learning, thinking and behavior. By estimation, 5.3 million Americans of all ages suffered from AD in 2015. This number is expected to increase to 16 million people by 2050. AD is the only cause of death in the top 10 of Americans that cannot be cured, prevented or slowed. Presently, there is no cure for AD, but early detection may help to figure out the root of AD mechanisms and improve the quality of life for patients who suffer from AD. In recent years, analysis of neuroimaging data has attracted a lot of interest with the recent improvements for early and accurate detection of AD. Neuroimaging techniques have become an important field of research due to the progress in their acquisition, storage and management in a wide range of applications including AD detection. High accurate image-based early detection of AD could provide valuable support for clinical treatments. High-dimensional classification methods have been a major target in the field of machine learning for the automatic AD detection. One major issue of automatic AD classification is the feature-selection method from high-dimensional feature space. This study proposes novel feature selection methods for high dimensional pattern recognition problem aimed at high accurate detection of AD, which uses the information from three dimensional magnetic resonance imaging (MRI) data extracted from the brain. MRI-based brain data used in the present study are obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI). This work focuses on structural MRI data and investigates extraction and selection of features, which are the main blocks in an automatic diagnosis detection system. In this regard, Voxel-based-morphometry (VBM) analysis of cross-sectional 3-Tesla 3D T1-weighted MRI data is utilized to perform feature extraction. VBM is an automated technique for assessment of whole brain structure with voxel-by-voxel comparisons which has been developed to analyze tissue concentrations or volumes between subject groups to distinguish degenerative diseases with dementia. The significant local differences in gray matter volumes (gray matter atrophies) based on VBM analysis are selected as 3-D volumes of interests (VOIs). Feature extraction based on the 3D voxel clusters detected by VBM on structural MRI (sMRI) and voxel values of VOIs are considered as raw features. In the feature selection stage, novel methods based on probability distribution function (PDF) and feature ranking are introduced to select most discriminative features from high-dimensional data. In the PDF-based feature selection approach, a novel statistical feature-selection process is employed, utilizing the PDF of the VOI to represent statistical patterns of the respective high dimensional sMRI sample. PDF of the VOIs can be considered a lower-dimensional feature vector representing sMRI images. The dimensionality of the PDF-based feature vector can be adjusted by changing the number of bins of the PDF. In this regard, the Fisher Criterion is used to determine the optimal number of bins of the histogram generating the PDF. In the proposed feature ranking method, all raw features are ranked using seven different statistical measures methods, namely, statistical dependency (SD), mutual information (MI), information gain (IG), Pearson’s correlation coefficient (PCC), t-test score (TS), Fisher’s criterion (FC), and the Gini index (GI). These measures are indicators of class separability, therefore the features with higher scores are assumed to be more discriminative. Hence it is critical to determine the number of top features. In the current study, to determine the number of top features, two methods namely, Fisher criterion and classification error are introduced. The Fisher Criterion between AD and HC groups is calculated for all sizes of feature vectors, where the vector size maximizing Fisher Criterion is selected as the number of top discriminative features. In a similar spirit, the estimated classification error on training set made up of the AD and HC groups is calculated. The vector size that minimizing this error is selected as the size of the top discriminative feature vector. In the classification stage, the support vector machine (SVM) classifiers with linear and non-linear kernels are employed to perform binary classification using 10 fold cross validation between patients who suffer from AD and age-matched healthy controls. Moreover, data fusion techniques are proposed to achieve higher performance in AD detection. In this regard, data fusion is introduced to improve the classification performance, by combining scores or vectors received from clusters obtained from MRI images based on the severity of gray matter atrophy in the brain. In addition, a novel data fusion approach among feature ranking methods is introduced. The results indicate that proposed approaches are reliable techniques that are highly competitive with the state-of-the-art techniques in classification of AD. Keywords: Alzheimer’s disease, Structural MRI, Voxel-based morphometry, Statistical feature extraction, Probability distribution function, Feature ranking, Fisher Criterion, classification error, Data fusion, , Support vector machine.
De-noising of Hyper-spectral Images in Wavelet Domain with Improved Soft Thresholding
A hyper-spectral image can be corrupted by noise during the transmission process. The noise does not have positive effect on the image, so it is essential to discard the noise before performing analysis to improve the quality of an image. Noise removal is among the important and challenging works for scientists and researchers in the field of image processing. The main objective of noise removal is to enhance the visual quality of the noisy using de-noising techniques. That is why researchers try to discard the noise before they perform further analysis. The main focus of the thesis is removing noise from hyper-spectral remote sensing images. Image de-nosing helps us improve the quality of the image, so we are able to analyze the image properly. In this thesis, we use 2D and 3D-DWT combined with hard and soft thresholding for de-noising hyper-spectral images. De-noising based on DWT introduces weakness such as lack of translation invariance. That is why; we suggest using Un-decimated Wavelet Transform (UWT) which discards the mentioned problem. Additionally, 2D and 3D-UWT with soft and hard thresholding functions were used as part of the proposed de-noising techniques. Finally we propose to use a new method for image de-noising in wavelet domain based on applying a smooth nonlinear soft threshold function on Un-decimated Wavelet Transform. This higher order threshold function is known as the improved soft thresholding function. Here we combined this function with 2D and 3D-UWT. Comparing the performance analysis between 2D-UWT and 3D-UWT using improved soft threshold function shows that 3D version outperforms 2D in terms of PSNR value and visual quality. This technique provides us with higher quality and improvement in PSNR value in comparison with several other methods available for de-noising. The proposed method achieves PSNR improvement by 2.12 dB for band 25 of Indian Pine, 1.29 dB for Cuprite Mining District image, 1.46 dB for Arizona Mining and 1.17dB for Golf of Mexico over de-noising based on 3D-UWT with standard soft thresholding technique. Keywords: Hyperspectral image de-noising, wavelet transform, hard and soft thresholding.
Markov Localization of an Indoor Quadcopter using Deep Learning
Localization is among one of the interesting subjects in robotics and can be spread from Unmanned Ground Vehicles (UGVs) to aerial ones. It is a point of interest for instance to localize robots in a warehouse or within an open area to define specific tasks. Unmanned Aerial Vehicles are also being used vastly indoors with GPSdenied environments. There are many localization methods recently being used in industry and research as such as Ultra-Wide Band (UWB), Bluetooth and (Global Positioning System) GPS. They have their own point of application in industry depending on their specifications. One of the best solutions is UWB with the least number of errors. In this thesis, we implemented a localization method based on Deep Learning. 16 patterns on the floor are used to make a specific map for localization. The proposed Deep Learning algorithm were able to detect each pattern correctly with 100% accuracy using majority voting for decision making in 3 seconds. The detection is performed real-time with the video feed of 30fps. Training and testing the network is done on Mobilenet which is based on Fast R-CNN deep learning architecture. All the processes are done on the quadcopter itself from navigation, control, and deep pattern detection using a single embedded computer. The quadcopter is equipped with a Raspberry Pi, Google Edge TPU embedded device with a flight controller in addition to a tracking and an RGB camera. The whole decision making of the patterns is performed via the embedded device connected to the Raspberry Pi in 30 fps and no pattern recognition process is employed on the ground computer. The drone odometry data is acquired via an Intel Realsense camera which provides IMU data to the drone. Only the codes for simple movements over the map have been sent to the drone from the ground station. Heading data is also provided by the tracking camera mounted on the quadcopter. Markov weights and the final decision weights have 100% confidence after each random path has been travelled over by the quadcopter. The drone was able to localize itself as a kidnapped robot, after flying over an average of two or maximum three patterns.
Gabor Wavelet Based Diabetic Retinopathy Detection Using Deep Learning
Diabetic Retinopathy (DR) is the disease that causes blindness when it reaches to the proliferative stage. There are four stages in DR namely: NO DR, Mild DR, Severe DR and Proliferative DR. The detection of diabetic retinopathy in the early stages can prevent sight loss in a notable amount of the population worldwide. The earliest signs of diabetic retinopathy are hemorrhages and hard exudates which are red and yellow lesions on the retina of the eye. Diagnosis of the DR is performed using retinal image analysis. Manual Analysis of these images to decide the presence of the DR is somewhat slow and costly. To ease the job of a medical consultant, we can take the images of the retina and feed them to the trained machine learning model and get the results whether the person has diabetic retinopathy or not. In this thesis, we propose a system to detect and classify diabetic retinopathy using the Gabor filter-based CNN with fusion. Proposed model fuses the decisions generated by Gabor-based CNN and traditional CNN pipelines. A preprocessing stage is employed to normalize images before feeding them into the CNN architecture. Preprocessing includes cropping and image resizing. A pre-trained CNN model VGG16, is selected as a deep machine learning architecture throughout the thesis. Decision fusion approaches including Sum Rule and Product Rule are employed throughout the fusion process. The recall, specificity, f1score, precision, and accuracy of the models have been studied and generated results are compared with each other and methods in the literature. The receiver characteristics curve (ROC) and Area under the curve (AUC) metrics are also measured to evaluate the classification performance. Messidor retina image database have been used to measure the performance of the iv proposed system. The results show that the proposed system to detect and classify diabetic retinopathy using the Gabor filter-based CNN with fusion generates higher performance over the state-of-the-art alternative methods in the literature. Keywords: Convolutional Neural Networks, Deep Learning, Machine Learning, Gabor Filters, Data Fusion.
Detection of Alzheimer’s Disease using 3D MRI Based on Key Slices Selected
Alzheimer’s disease (AD) is one of the most common irreversible dementia disease affecting mostly old people specially in older adulthood. This disease corresponds to a particular group of aging dementia which deteriorates long-term and short-term memory, behavior and thinking. Currently there is no treatment for AD, but early detection of AD can help finding out mechanisms of AD and make better life for patients who suffer from AD. In this study we detect Alzheimer’s disease (AD) subjects among elderly cohorts including health and AD subjects. One of the main issues of automatic AD classification is feature extraction in high-dimensional feature space. This thesis proposes new feature extraction methods for high-dimensional pattern recognition problem aimed at accurate detection of AD. The proposed methods use information from three dimensional magnetic resonance imaging (MRI) brain data with 2D slices in three orthogonal directions. The proposed method includes the calculation of Fisher Criterion between the AD and HC groups in order to select key-slices in the coronal, sagittal and axial directions. The preprocessing phase involve region detection to segment region of interest (ROI) based on displacement field (DF) method. Then we utilized energy, contrast and homogeneity metrics along with feature vectors generated by PCA and probability distribution function (PDF) methods in feature extraction phase for each key-slice selected in the earlier phase. Features coming from each key-slice are combined through feature fusion for improved accuracy. Experimental results show fusion method that used with brain mask give us the higher or comparable results compared with other feature extraction techniques in the literature. Keywords: Alzheimer’s disease, MRI, region of interest, Statistical feature extraction, data fusion, classification, support vector machine