(50 minutes to learn)
Gradient descent, also known as steepest descent, is an iterative optimization algorithm for finding a local minimum of differentiable functions. At each iteration, gradient descent operates by moving the current solution in the direction of the negative gradient of the function (the direction of "steepest descent").
This concept has the prerequisites:
- Be able to apply gradient descent to functions of several variables
- Why is gradient descent not guaranteed to find the global optimum?
- Why is gradient descent guaranteed to converge? What can we say about the solution it obtains?
Core resources (read/watch one of the following)
→ Convex Optimization
A graduate-level textbook on convex optimization.
Location: pages 467 - 475
→ Coursera: Machine Learning (2013)
Location: Article: Gradient Descent
Supplemental resources (the following are optional, but you may find them useful)
→ Coursera: Machine Learning
→ Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation