Change points

Joseph Hall (October 2019)

This notebook demonstrates the use of the ChangePoints kernel, which can be used to describe one-dimensional functions that contain a number of change-points, or regime changes. The kernel makes use of sigmoids (\(\sigma\)) to blend smoothly between different kernels. For example, a single change-point kernel is defined by:

\begin{equation} \textrm{cov}(f(x), f(y)) = k_1(x, y)\cdot\bar{\sigma}(x, y) + k_2(x, y)\cdot\sigma(x, y) \end{equation}

where \(\sigma(x, y) = \sigma(x)\cdot\sigma(y)\) and \(\bar{\sigma}(x, y) = (1 - \sigma(x))\cdot(1 - \sigma(y))\). The sigmoid (\(\sigma\)) is parameterized by a location (\(l\)) and a width (\(w\)).

import matplotlib.pyplot as plt
import numpy as np

import gpflow

np.random.seed(123)  # for reproducibility of this notebook

plt.style.use("ggplot")
%matplotlib inline


def plotkernelsample(k, ax, xmin=-3, xmax=3, title=None):
    xx = np.linspace(xmin, xmax, 100)[:, None]
    ax.plot(xx, np.random.multivariate_normal(np.zeros(100), k(xx), 3).T)
    ax.set_title(title)

2024-02-07 11:46:07.357559: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2024-02-07 11:46:07.402672: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-02-07 11:46:07.402715: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-02-07 11:46:07.403955: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-02-07 11:46:07.411180: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2024-02-07 11:46:07.411751: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

2024-02-07 11:46:08.627974: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

We demonstrate the use of the kernel by drawing a number of samples from different parameterizations. Firstly, a simple single change-point between two kernels of differing lengthscales.

np.random.seed(1)

base_k1 = gpflow.kernels.Matern32(lengthscales=0.2)
base_k2 = gpflow.kernels.Matern32(lengthscales=2.0)
k = gpflow.kernels.ChangePoints([base_k1, base_k2], [0.0], steepness=5.0)

f, ax = plt.subplots(1, 1, figsize=(10, 3))
plotkernelsample(k, ax)

Secondly, an implementation of a “change window” in which we change from one kernel to another, then back to the original.

np.random.seed(3)

base_k1 = gpflow.kernels.Matern32(lengthscales=0.3)
base_k2 = gpflow.kernels.Constant()
k = gpflow.kernels.ChangePoints(
    [base_k1, base_k2, base_k1], locations=[-1, 1], steepness=10.0
)

f, ax = plt.subplots(1, 1, figsize=(10, 3))
plotkernelsample(k, ax)

And finally, allowing different change-points to occur more or less abruptly by defining different steepness parameters.

np.random.seed(2)

base_k1 = gpflow.kernels.Matern32(lengthscales=0.3)
base_k2 = gpflow.kernels.Constant()
k = gpflow.kernels.ChangePoints(
    [base_k1, base_k2, base_k1], locations=[-1, 1], steepness=[5.0, 50.0]
)

f, ax = plt.subplots(1, 1, figsize=(10, 3))
plotkernelsample(k, ax)