Learning track

Sequence Models

Follow the path from recurrent state to encoder-decoder models and attention, the immediate ancestors of the Transformer.

core · 5 available lessons

Sequence Models

Follow the path from recurrent state to encoder-decoder models and attention, the immediate ancestors of the Transformer.

  1. Modelling sequences: the setup
  2. RNNs and the vanishing gradient
  3. LSTM and GRU: gates that remember
  4. Seq2seq, encoder-decoder, and the bottleneck
  5. Bahdanau attention: the idea that changed everything