In the translation task, a sentence x = x 1 , I {\displaystyle \mathbf {x} =x_{1,I}} (consisting of I {\displaystyle I} tokens x i {\displaystyle x_{i}} ) in the source language is to be translate… 展开

Early approaches
In 1987, Robert B. Allen demonstrated the use of feed-forward neural … 展开

NMT has overcome several challenges that were present in statistical machine translation (SMT): 1. NMT's full reliance on continuous representation of tokens overcame sparsi… 展开

Cross-entropy loss
NMT models are usually trained to maximize the likelihood of observing the training data. I.e., for a dataset of T {\displaystyle T} … 展开

As outlined in the history section above, instead of using an NMT system that is trained on parallel text, one can also prompt a generative LLM to translate a text. These models di… 展开