Ön-eğitimli dil modelleri ile Türk işaret dili için işaret dili çevirisi

Sign Language Translation (SLT) is a machine learning task that aims to provide accessibility systems for the deaf and hard-of-hearing community. With the recent advances within the area, SLT technologies have shifted towards large-scale training with pretrained large language models (LLMs). However, most technologies and datasets are provided for American Sign Language (ASL) and British Sign Language (ASL). To achieve similar advances for Turkish Sign Language (TID) translation, we are proposing the first large-scale open-source translation dataset for TID, comprising over 500 hours of video footage and aligned Turkish translations by utilizing publicly available YouTube content. In this work, our dataset provides a scalable and reproducible framework for SLT research. We propose a comprehensive translation system with various inter-changeable components designed for SLT on an agglutinative language to compensate for the learning alignment between both low-resource languages, namely TID and Turkish. This body of work compares feature extraction strategies (Linear layers, Multi-layer Perceptrons, and Convolutional encoders), model sizes, and their pretraining approaches (multilingual versus Turkish-specific) to the effect on sign translation performance. Innovative training and evaluation techniques, including paraphrasing-based reference generation and sign language-specific augmentations, are also introduced for developing an SLT benchmark for TID.