User Guide

You can use this document to get familiar with the more in-depth topics of GPflow. If you are new to GPflow you should see our Getting Started guide first. We have also provided a flow diagram to guide you to the relevant parts of GPflow for your specific problem.

Theoretical notes

The following notebooks relate to the theory of Gaussian processes and approximations. These are not required reading for using GPflow, but are included for those interested in the theoretical underpinning and technical details.

• Derivation of VGP equations
• Derivation of SGPR equations
• Conjugate Gradient Lower Bound
• Discussion of the GP marginal likelihood upper bound

• Comparing FITC approximation to VFE approximation

Why we like the Variational Free Energy (VFE) objective rather than the Fully Independent Training Conditional (FITC) approximation for our sparse approximations.

• Sanity checking when model behaviours should overlap

Demonstrates the overlapping behaviour of many of the GPflow model classes in special cases (specifically, with a Gaussian likelihood and, for sparse approximations, inducing points fixed to the data points).

Tailored models

This section shows how to use GPflow’s utilities and codebase to build new probabilistic models. These can be seen as complete examples.

• Kernel Design

How to implement a covariance function that is not available by default in GPflow. For this example, we look at the Brownian motion covariance.

• Mixing TensorFlow models with GPflow

Two ways to combine TensorFlow neural networks with GPflow models.

• Custom mean functions: metalearning with GPs

How to use a neural network as a mean function.

• Mixture Density Networks in GPflow

How GPflow’s utilities make it easy to build other, non-GP probabilistic models.

Advanced needs

This section explains the more complex models and features that are available in GPflow.

• MCMC (Markov Chain Monte Carlo)

Using Hamiltonian Monte Carlo to sample the posterior GP and hyperparameters.

• Ordinal regression

Using GPflow to deal with ordinal variables.

• Stochastic Variational Inference for scalability with SVGP

Using GPflow’s Sparse Variational Gaussian Process (SVGP) model [HMG15] [HFL13]. Use sparse methods when dealing with large datasets (more than around a thousand data points).

• Multiclass classification

On classification with more than two classes.

• Gaussian process regression with varying output noise

Most GP models assume the noise is constant across all data points. This notebook shows how to model simple varying noise.

• Heteroskedastic Likelihood and Multi-Latent GP

This is a more expensive, but also more powerful way to handly noise that varies across data points.

• Change points

How to deal with regime-changes in 1D datasets.

• Convolutional Gaussian Processes

How we can use GPs with convolutional kernels for image classification.

• Multi-output Gaussian processes in GPflow

For when you have multiple outputs, that all are observed at all data points.

• A simple demonstration of coregionalization

For when you have multiple outputs, but not all of them are observed at every data point.

• Faster predictions by caching

How to use caching to speed up repeated predictions.

• Bayesian Gaussian process latent variable model (Bayesian GPLVM)

How to use the Bayesian GPLVM model. This is an unsupervised learning method usually used for dimensionality reduction.

• Variational Fourier Features in the GPflow framework

In this notebook we demonstrate how new types of inducing variables can easily be incorporated in the GPflow framework. As an example case, we use variational Fourier features.

• Natural gradients

How to optimize the variational approximate posterior’s parameters.