the kernel trick
(50 minutes to learn)
We can use linear models to model complex nonlinear functions by mapping the original data to a basis function representation. Such a representation can get unweildy, however. The kernel trick allows us to implicitly map the data to a very high (possibly infinite) dimensional space by replacing the dot product with a more general inner product, or kernel.
This concept has the prerequisites:
Core resources (read/watch one of the following)
→ Gaussian Processes for Machine Learning
A graduate-level machine learning textbook focusing on Gaussian processes.
Location: Section 2.1, pages 7-12
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Sections 6-6.1, pages 291-294
Supplemental resources (the following are optional, but you may find them useful)
→ Coursera: Machine Learning (2013)
An online machine learning course aimed at a broad audience.
Location: Lecture "Kernels I"
- Click on "Preview" to see the videos.
→ Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation