From prompts to precision: A comparative experimental study of translation quality outputs with focus on role-based (Persona) prompting for medical texts

With the advancements in artificial intelligence-driven machine translation activities, designing prompts for large language models for translation task is an area that recently has gained interest in translation literature. The effectiveness of assigning personas into prompts, however, is a subject that needs further research in medical/pharmaceutical texts domain. In response to this gap, this study aims to evaluate the impact of offering role-based (persona) prompts to large language models. By doing so, the research assesses the quality of translation outputs of role-prompted large language models against 3 different source texts with pharmaceutical content, while comparing these outputs to zero-shot prompted models alongside the outputs from long-rooted conventional neural machine translation system: Google Translate. While evaluating automated metric scores such as BLEU (Bilingual Evaluation Understudy) and COMET (Crosslingual Optimized Metric for Evaluation of Translation) from the machine translation outputs, I performed TQE (Translation Quality Evaluation) analysis through a customized TQE error-typology tool, motivated and inspired by MQM (Multidimensional Quality Metrics) framework. Therefore, this study tries to discover how effective role-based prompts are in medical/pharmaceutical domain translation with both a quantitative and also a qualitative approach, as I believe that such a hybrid approach is essential for a solid oversight of quality. Lastly, the study will include discussions over the widely-discussed concepts of machine translation ethics, translator competence, and human-machine collaboration.