recurrent neural networks
(3.2 hours to learn)
Recurrent neural networks (RNNs) are a kind of neural net often used to model sequence data. They maintain a hidden state which can "remember" certain aspects of the sequence it has seen. RNNs can be trained using backpropagation through time, although efficient training remains an open problem.
This concept has the prerequisites:
- Understand what a recurrent neural net is and how it can be used to model sequence data.
- Know how to train an RNN using backpropagation through time.
- Be aware of why training RNNs with backpropagation is unstable.
Core resources (read/watch one of the following)
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
- It may be helpful to first skim Lecture4, which covers sequence modeling.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation