In the translation task, a sentence x = x 1 , I {\displaystyle \mathbf {x} =x_{1,I}} (consisting of I {\displaystyle I} tokens x i {\displaystyle x_{i}} ) in the source language is to be tra… See more

Early approaches
In 1987, Robert B. Allen demonstrated the use of feed-forward neural … See more

NMT has overcome several challenges that were present in statistical machine translation (SMT): 1. NMT's full reliance on continuous representation of tokens overcame s… See more

Cross-entropy loss
NMT models are usually trained to maximize the likelihood of observing the training data. I.e., for a dataset of T {\displaysty… See more

As outlined in the history section above, instead of using an NMT system that is trained on parallel text, one can also prompt a generative LLM to translate a text. These mod… See more