I try the understand how the Gaussian Process works. So if I'm sampling from it, I will get values, which come from a multivariate distribution.

Let's say I have 5 input features in my training data. So one training point is a 5d vektor like: [x1, x2, x3, x4, x5]. Each training point has a 1d target value.

After inferring the posterior distribution and plotting samples from the GP, what actually is on the y-axis and what is on the X-axis?

Am I right that the y-value, so actually the function value, is my target ? Is it right that on the x-axis there are indices, which stand for one training point, so here: [x1,x2,x3,x4,x5]?


1 Answers

maw501 On

It's easier to deal with all your questions together though it would be helpful if you could provide an example of the type of charts you are referring to.

The x-axis is not usually the variable index and (slightly guessing where the confusion is coming from) this way of thinking is usually only mentioned to explain how it is possible to view samples from a large dimensional Gaussian in a single plot. For example charts like the below are common in introductions:

enter image description here

Where the above was constructed by calculating the covariance matrix between 5 observations (a 5-dimensional Gaussian) and sampling from it. This way of visualizing the function values is how we are able to represent multi-dimensional Gaussians on a single plot. However this view imagines ordering the points according to their index in the covariance matrix which doesn't really make sense but helps understand the different way of visualization (which is helpful for understanding Gaussian processes).

In general, the x-axis will be the values of X for which we wish to calculate a point at and thus can take on any real-value. For data with 1 input dimension (d=1 and equal to time, say) this would result in many of the charts that are common when introducing Gaussian processes with an appeal to non-linear regression. However once d>1 even this view of things breaks down as the input space is now a high-dimensional grid (arbitrarily fine if we wish) for which we can calculate function values at.

Nevertheless, when d=1 this way of visualizing helps us plot multi-dimensional Gaussians a lot easier and is helpful for explaining Gaussian processes.

The y-axis is generally the function value (i.e. the target).

Edit: I was going to give a much fuller reply but realised stackoverflow doesn't support latex. If you would like to read more I've written about this point on my blog with more background context than the above.