Findings of the Quality Estimation Shared Task at WMT 2024

This technical report presents the findings from the Quality Estimation (QE) shared task conducted at the Ninth Conference on Machine Translation (WMT) in 2024. The primary objective of the shared task is to evaluate the quality of neural machine translation outputs without relying on reference translations. The report outlines the methodologies employed, including the assessment of sentence-level quality scores and the detection of error spans. It also introduces a new automated post-editing (APE) task aimed at improving translation quality. The document details the creation of new test sets with human annotations for various language pairs, including English to German, Spanish, and Hindi, as well as assessments for Indic languages. Furthermore, the report analyzes the performance of different models in relation to various linguistic phenomena and biases. The findings contribute to the ongoing development of reliable QE systems and provide insights into the effectiveness of different approaches in the context of machine translation.