ridge regression as SVD
(1.1 hours to learn)
It's possible to write the ridge regression solution in terms of the SVD of the dataset. This gives insight into how it makes predictions. It also gives a way of defining the "degrees of freedom" or "effective number of parameters" of the model, which lets us analyze the degree of overfitting.
This concept has the prerequisites:
Core resources (read/watch one of the following)
→ The Elements of Statistical Learning
A graudate-level statistical learning textbook with a focus on frequentist methods.
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 7.5.3, pages 228-230
- If you don't know what PCA is, just think of it as the SVD.
-No Additional Notes-
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation