Abstract
Multi-output Gaussian processes (MOGPs) leverage the flexibility andinterpretability of GPs while capturing structure across outputs, which isdesirable, for example, in spatio-temporal modelling. The key problem withMOGPs is the cubic computational scaling in the number of both inputs (e.g.,time points or locations), n, and outputs, p. Current methods reduce this toO(n^3 m^3), where m < p is the desired degrees of freedom. This computationalcost, however, is still prohibitive in many applications. To address thislimitation, we present the Orthogonal Linear Mixing Model (OLMM), an MOGP inwhich exact inference scales linearly in m: O(n^3 m). This advance opens up awide range of real-world tasks and can be combined with existing GPapproximations in a plug-and-play way as demonstrated in the paper.Additionally, the paper organises the existing disparate literature on MOGPmodels into a simple taxonomy called the Mixing Model Hierarchy (MMH).