This roadmap gives the background for my ICML 2013 paper, "Structure discovery in nonparametric regression through compositional kernel search." This covers the basic machine learning concepts that the paper depends on, and should be sufficient for understanding it at a conceptual level.
The paper focuses in particular on learning the structure of Gaussian processes. In particular, you'll want to be familiar with:
- using Gaussian processes for nonparametric regression
- how to construct covariance kernels for Gaussian processes. The single most useful reference is probably chapter 4 of Gaussian Processes for Machine Learning.
- learning the hyperparameters of a kernel
To control for the complexity of the kernel, we use the Bayesian information criterion (BIC).