MCL-MT: Multi-Level Contrastive Learning for Many-to-many Multilingual Neural Machine Translation

Abstract

Currently,   mainstream  multilingual  translation  models  are  trained  mainly  on  English- related language pairs.   These  systems usually perform well on the English-related direc- tions, known as supervised directions, while the  translation  performance  on  non-English directions (zero-resource directions) is weak. Therefore, a method called mRASP2 has been proposed which integrates monolingual corpus and bilingual corpus under a unified training framework  through  contrastive  learning  and alignment  enhancement  methods,  so  that  it can make  full use  of the  corpus,  learn bet- ter language-independent representations, and thus improve the performance of multilingual translation. In this paper, we propose a method to train an NMT model. Unlike the sentence- level alignment used in most previous  stud- ies, this paper uses MCL-MT to explicitly in- tegrate  word-level  information  of  each  pair of parallel sentences into contrastive learning. English-centric translation directions show su- perior performance with MCL-MT’s method- ology  in  comparison  to  the  pretrained  and fine-tuned  model  mBART.  On  non-English translations, MCL-MT offers an average 10+ BLEU score increase when compared to the multilingual Transformer baseline.