the evidence approximation
(1 hours to learn)
The evidence approximation is an approximation to Bayesian parameter estimation and model comparison. Rather than integrating out model hyperparameters, the hyperparameters chosen to maximize the marginal likelihood of the data.
This concept has the prerequisites:
Core resources (read/watch one of the following)
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 5.6, pages 172-176
- Bayesian parameter estimation: Gaussian distribution
Supplemental resources (the following are optional, but you may find them useful)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Sections 3.5-3.5.2, pages 165-169
- Bayesian linear regression
- The evidence approximation requires integrating out the model parameters. Variational Bayes gives a way of doing this.
- An alternative to the evidence approximation is to define a prior over the hyperparameters and integrate them out. This is an instance of hierarchical Bayesian modeling .
- The evidence approximation can be computed exactly in the case of Gaussian processes.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation