DCU ADAPT Submission for WMT24 Multimodal Translation

This technical report details the system description of DCU_NMT's submission to the WMT-WAT24 English-to-Low-Resource Multimodal Translation Task, focusing on the English-to-Hindi track. The report outlines the development of both text-only and multimodal neural machine translation (NMT) systems. The text-only systems were trained from scratch using constrained data and augmented with back-translated data. For the multimodal approach, a context-aware transformer model was implemented, integrating visual features as additional contextual information. The report presents the datasets used, including the Visual Genome datasets for Hindi, Bengali, Malayalam, and Hausa, and describes the experimental details for both translation tasks. Results indicate that the multimodal system, trained on limited data, showed improvements over the text-only baseline in evaluation sets, suggesting the potential benefits of incorporating visual information into translation processes. The findings are significant for enhancing translation accuracy in low-resource language contexts.