Disentangled representation learning in isolated sign language recognition

Representation learning is an essential part of all deep learning tasks. Achieving good performance in recognition, generation, and classification heavily depends on learning meaningful and reliable representations. It is important to gather informative representations that are not affected by unnecessary details in all cases. Sign Language Recognition is one of the areas where deep learning models have been successfully used. Sign Language Recognition (SLR) is essential to exchange information between those who know sign language and those who do not. The input of an SLR model is a video in which an individual performs a sign or multiple signs. Therefore, Convolutional Neural Networks (CNN) are commonly a part of deep learning-based SLR frameworks. However, CNN-based recognition frameworks tend to capture the characteristics of the identity in the foreground, such as face attributes, hand and body shape, and skin color. This challenge is often encountered in problems such as face and gait recognition, image manipulation, and person re-identification problems. In this thesis, a disentangled representation learning framework is proposed to separate the latent factors in the sign and signer representations and eliminate the irrelevant identity information to improve sign recognition performance. Various disentanglement techniques, including regularized adversarial training, are investigated. Experiments are conducted on two isolated Turkish sign language benchmark datasets. The effect of feature disentanglement and its potential to improve recognition performance are discussed with qualitative and quantitative analysis.