learning invariances in neural nets

(40 minutes to learn)


The human visual system is capable of recognizing objects despite changes in factors such as location, orientation, and lighting. We'd like the representations learned by neural networks to be invariant to at least some of these things as well. There are several different strategies for achieving this, including enforcing invariance in the network architecture, using an appropriate regularization term, or generating randomly perturbed training data.


This concept has the prerequisites:

Core resources (read/watch one of the following)


Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
  • Lecture "Why object recognition is difficult"
  • Lecture "Achieving viewpoint invariance"
Author: Geoffrey E. Hinton


See also