convolutional neural nets
(1.3 hours to learn)
Convolutional neural networks are a kind of feed-forward neural net architecture geared towards visual processing. In each layer, there are several groups of units whose weights are repeated (or "shared") across all spatial locations. The forward pass and backpropagation updates can both be computed efficiently using convolution.
This concept has the prerequisites:
Core resources (read/watch one of the following)
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
- Lecture "Convolutional nets for digit recognition"
- Lecture "Convolutional nets for object recognition"
→ Gradient-based learning applied to document recognition
Supplemental resources (the following are optional, but you may find them useful)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 5.5.6, pages 267-269
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 16.5.1, pages 564-568
- Convolutional nets are one way to get invariances in a neural net representation. Other ways include:
- augmenting the training set with warped examples
- tangent propagation , which penalizes instability with respect to transformations
- Tikhonov regularization , which penalizes instability with respect to noise
- Convolutional neural nets learn oriented filters similar to the receptive fields in the human primary visual cortex .
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation