recurrent neural networks
(3.2 hours to learn)
Summary
Recurrent neural networks (RNNs) are a kind of neural net often used to model sequence data. They maintain a hidden state which can "remember" certain aspects of the sequence it has seen. RNNs can be trained using backpropagation through time, although efficient training remains an open problem.
Context
This concept has the prerequisites:
- backpropagation (RNNs are trained using backpropagation.)
Goals
- Understand what a recurrent neural net is and how it can be used to model sequence data.
- Know how to train an RNN using backpropagation through time.
- Be aware of why training RNNs with backpropagation is unstable.
Core resources (read/watch one of the following)
-Free-
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
Other notes:
- It may be helpful to first skim Lecture4, which covers sequence modeling.
See also
- Other models of sequence data include: