Jeremy Jordan (@jeremyjordan.me)

new blog post: all about transformers! covering the intuition behind an "attention only" network, building up an understanding of scaled dot product attention, and detailing the original transform architecture. let me know what you think! 🙏

loading . . .

Understanding the Transformer architecture for neural networks The attention mechanism allows us to merge a variable-length sequence of vectors into a fixed-size context vector. What if we could use this mechanism to entirely replace recurrence for sequential modeling? This blog post covers the Transformer architecture which explores such an approach. https://www.jeremyjordan.me/transformer-architecture/