(1.6 hours to learn)
Probabilistic principal component analysis (PCA) is a formulation of PCA as a latent variable model. Each data point is assumed to be generated as a linear function of Gaussian latent variables, plus noise. Like PCA, it has a closed form solution in terms of the truncated SVD of the covariance matrix.
This concept has the prerequisites:
- principal component analysis
- computations on multivariate Gaussians (Probabilistic PCA involves performing inference in a model involving multivariate Gaussians.)
- maximum likelihood (Fitting probabilistic PCA is done using maximum likelihood.)
- principal component analysis (proof) (The maximum likelihood solution is derived using a variation of the proof of PCA correctness.)
- optimization problems (Finding the maximum likelihood solution requires solving an optimization problem.)
Core resources (read/watch one of the following)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Sections 12.2-12.2.1, pages 570-577
Supplemental resources (the following are optional, but you may find them useful)
→ Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
Location: Section 21.4, page 436
- factor analysis
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 12.2.4, pages 395-396
- Probabilistic latent semantic analysis (pLSA) is another probabilistic model similar to PCA, but better geared toward discrete data.
- Bayesian PCA is a Bayesian version of pPCA.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation