Findings of the WMT24 General Machine Translation Task

This technical report presents the findings from the General Machine Translation (MT) Task conducted during the 2024 Conference on Machine Translation (WMT). The task involved participants building machine translation systems for 11 language pairs, evaluated across diverse domains such as news, social content, speech, and literary texts. The report outlines the methodology, including the collection of test data and the evaluation process using a new protocol called Error Span Annotations (ESA). It details the challenges faced in evaluating translation capabilities, particularly in the speech domain, and highlights the performance of various systems, including those utilizing large language models (LLMs). The report also discusses the results of the evaluation, noting that human references outperformed many automatic evaluations. Additionally, it includes insights into the number of participants and the types of data used for testing, emphasizing the importance of maintaining high-quality source texts to ensure accurate evaluation outcomes.