MetricX-24 Submission to WMT 2024 Metrics Shared Task

This paper is a technical report detailing the MetricX-24 submissions to the WMT 2024 Metrics Shared Task. It outlines the advancements made over the previous version, MetricX-23, particularly focusing on a hybrid reference-based/-free metric capable of scoring translations with or without reference segments. The report describes the training methodology, which involves a two-stage process utilizing direct assessment (DA) ratings and MQM ratings, augmented with synthetic examples to enhance robustness against common failure modes in translation evaluation. Key findings include the effectiveness of synthetic data in improving performance, particularly in identifying under-translation and unrelated translations. The paper also discusses the evaluation data used, including MQM and DA ratings, and the construction of synthetic datasets aimed at addressing specific translation issues. The results indicate a significant performance increase over prior metrics, demonstrating the importance of training on diverse data sources to improve translation quality assessment.