In the translation task, a sentence $${\displaystyle \mathbf {x} =x_{1,I}}$$ (consisting of $${\displaystyle I}$$ tokens $${\displaystyle x_{i}}$$) in the source language is to be translated into a sentence 展开

Early approaches
In 1987, Robert B. Allen demonstrated the use of feed-forward neural networks for translating auto-generated English sentences with a limited … 展开

Cross-entropy loss
NMT models are usually trained to maximize the likelihood of observing the training data. I.e., for a … 展开

• Koehn, Philipp (2020). Neural Machine Translation. Cambridge University Press.
• Stahlberg, Felix (2020). Neural Machine Translation: A Review and Survey. 展开

NMT has overcome several challenges that were present in statistical machine translation (SMT):
• NMT's full reliance on continuous representation of tokens overcame sparsity issues caused by rare words or phrases. Models were … 展开

As outlined in the history section above, instead of using an NMT system that is trained on parallel text, one can also prompt a generative LLM to translate a text. These models differ from an encoder-decoder NMT system in a number of ways: 展开

• Attention (machine learning)
• Transformer (machine learning model)
• Seq2seq 展开