Measuring Translation Quality with Smartling

This technical report outlines Smartling's approach to measuring translation quality, focusing on the implementation of the Multidimensional Quality Metrics (MQM) framework. The report begins by addressing the challenges of ensuring consistent translation quality across various languages and workflows, emphasizing the importance of objective evaluation criteria to mitigate subjective assessments. It details the steps involved in Smartling's quality evaluation process, including the selection of locales, curation of sample sets, and the engagement of professional evaluators. The report explains how errors are categorized and scored according to severity, providing a structured methodology for assessing translations. Additionally, it discusses the flexibility of the MQM framework, which allows for the evaluation of both human and machine-generated translations. The document concludes with an overview of how the evaluation results inform action plans for improving translation quality and training machine-learning applications. This structured approach aims to enhance the reliability of translation services offered by Smartling.