The use of machine translation in high-stakes texts: A case study on guidelines for natural disasters

The rapidly evolving artificial intelligence and machine learning technologies have led to growing concern among translators regarding potential job displacement. In response to this concern, this study evaluates the performance of DeepL Translator and Google Translate in translating Turkish guidelines for natural disasters into English, and explores whether raw machine translation output can be used in such high-stakes contexts. Additionally, it investigates the effectiveness of pre-editing operations in correcting errors in machine translation outputs. Using the MQM (Multidimensional Quality Metrics) framework, a comprehensive analysis was conducted on the raw outputs of both machine translation systems. Then, pre-editing strategies were developed for each error. Findings reveal that while both systems produced fewer errors than expected, errors of "major" and "critical" severity levels were present in machine translation outputs, underscoring the unpredictable nature of black-box neural machine translation systems. Pre-editing was found to be effective in addressing a significant portion of errors, with the exception of errors in terminology category. In conclusion, this study emphasizes the indispensable role of human translators in the translation process, despite the advanced capabilities of current machine translation systems.