(1.1 hours to learn)
The gradient of a function gives the direction of maximum increase. Its entries are given by the partial derivatives of the function. It is widely used in optimization.
This concept has the prerequisites:
- linear approximation (The gradient gives another way to represent the linear approximation to a function.)
- partial derivatives (The gradient is computed in terms of partial derivatives.)
- dot product (The linear approximation is given in terms of a dot product with the gradient.)
- functions of several variables (The gradient is typically applied to functions of several variables)
Core resources (read/watch one of the following)
→ MIT Open Courseware: Multivariable Caclulus (2010)
Video lectures for MIT's introductory multivariable calculus class.
→ Multivariable Calculus
An introductory multivariable calculus textbook.
Location: Section 13.8, "Directional derivatives and the gradient vector," up to "The gradient vector as normal vector," pages 907-913
→ Multivariable Mathematics
A textbook on linear algebra and multivariable calculus with proofs.
Location: Section 3.4, "The gradient," pages 104-106
- See section 3.1 for the definition of directional derivatives.
Supplemental resources (the following are optional, but you may find them useful)
→ Khan Academy: Calculus
- Gradient ascent , or 'hill climbing', is a simple and widely used optimization algorithm.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation