Back to Timeline
2017-06-12PaperLandmark

Attention Is All You Need: The Transformer

Google researchers introduce the Transformer architecture, replacing recurrent networks with pure attention mechanisms. This paper is arguably the most influential in modern AI — it's the foundation of GPT, BERT, and virtually every large language model today.

References

  • Attention Is All You Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin

    NeurIPS2017