Awesome Transformer [](https://github.com/sindresorhus/awesome)

A curated list of NLP resources focused on Transformer.

What is Transformer?

Transformer is a novel neural network architecture which has shown great promise in many natural language processing(NLP) tasks such as machine translation, language modeling, question answering and reading comprehension.

More info here.

Table of Contents

• Implementations
• Theory
  • Lectures
  • Architecture Variants
  • Surveys
• Papers
  • Machine Translation
  • Language Modeling
  • Other
• Datasets
• Blogs

Implementations

repository	framework	organization
tensor2tensor	TensorFlow(official)	Google
allennlp	PyTorch	Allen AI
annotated-transformer	PyTorch	Harvard NLP
attention-is-all-you-need-pytorch	PyTorch	jadore801120
gluon-nlp	MXNet	dmlc
attention-is-all-you-need-keras	Keras	Lsdefine
attention_is_all_you_need	Chainer	soskek

Theory

Lectures

lecture	course	school
Transformers and Self-Attention For Generative Models	CS224n: Natural Language Processing with Deep Learning	Stanford
Self Attention/CNNs	CS 11-731 Machine Translation and Sequence-to-Sequence Models	CMU
Attention	CS 11-747 Neural Networks for NLP	CMU
Attention	CSC 421/2516 Neural Networks and Deep Learning	Toronto