Adapting Large Language Models for Low-Resource Translation

This document is a research article that investigates the adaptation of Large Language Models (LLMs) for low-resource translation tasks. It addresses the performance gap between LLMs and Neural Machine Translation (NMT) models in low-resource languages (LRLs). The authors explore the significance of two key factors: the role of parallel data and the impact of diversity in Supervised Fine-Tuning (SFT). The findings indicate that, contrary to trends observed in high-resource languages, parallel data is essential for both Continued Pre-Training (CPT) and SFT in low-resource contexts. Additionally, the study reveals that diversity in training data can lead to negative interference rather than beneficial transfer. The research includes experiments conducted on LLMs across Indigenous American and North-East Indian languages, demonstrating that focused multilingual fine-tuning with increased training epochs yields better results. The insights aim to enhance the effectiveness of LLMs in translating low-resource languages, contributing to the development of more inclusive multilingual models.