Abstract
This paper is an attempt to bridge the conceptual gaps between researchersworking on the two widely used approaches based on positive definite kernels:Bayesian learning or inference using Gaussian processes on the one side, andfrequentist kernel methods based on reproducing kernel Hilbert spaces on theother. It is widely known in machine learning that these two formalisms areclosely related; for instance, the estimator of kernel ridge regression isidentical to the posterior mean of Gaussian process regression. However, theyhave been studied and developed almost independently by two essentiallyseparate communities, and this makes it difficult to seamlessly transferresults between them. Our aim is to overcome this potential difficulty. To thisend, we review several old and new results and concepts from either side, andjuxtapose algorithmic quantities from each framework to highlight closesimilarities. We also provide discussions on subtle philosophical andtheoretical differences between the two approaches.