NARX Neural network prediction?

Question

I am trying to solve a time series problem using the NARX Neural Network solution that Matlab provides. I am trying to understand how to predict actual values, but the results I get are almost perfect! The errors are so small that I am not sure if I am actually predicting. I just want to make sure I am doing everything right!

Basically I train the network with some samples using the GUI solution. Then I use the following script to test the neural network with new samples:

      X = num2cell(open2(1:end))'; % input
      T = num2cell(close2(1:end))'; % this is the output I should get

      net = removedelay(net);
      [Xs,Xi,Ai,Ts] = preparets(net,X,{},T);

      Y = net(Xs,Xi,Ai);

      plotresponse(Ts,Y)
      view(net)

      Y = cell2mat(Y);
      T = cell2mat(T);

      sizey = length(Y);
      sizet = length(T);

      T = T(1:sizey);

      figure
      plot(1:sizey,T,1:sizey,Y)

The graph I am getting is almost identical to the original target time series function. The errors are really small and the only difference is that the graph (Y) is shifted 2 samples to the left. But, am I really predicting?

Here's part of the graph:

Thanks in advance!

Update: The actual prediction graph is shifted to the right and not to the left. The targets provided by the preparets function (blue) occurs before! So it doesn't show it's actually predicting.

Answer 1

Right Shift

Your graph shows a timeshift of 1 (not 2!) timestep(s). This is not ideal, but can happen when the delays are badly chosen which leads to this kind of delay pattern. (For further explanation have a look at this question on MATLAB CENTRAL. In fact, Greg Heath posted a lot of material on ANNs, very worth the read even though it's sometimes a bit short to be understood immediately, especially for beginners.) So, to avoid this you have to look into the correlation patterns of your data.

Removedelay()

Now, I'm assuming that you wanted to correct for this behaviour by removing the delay of the network instead. Unfortunately, this is not what removedelay() is meant for:

This example uses a timedelaynet, but can be adopted for NAR and NARX networks as well, and I found the description very helpful. In combination with a quote from removedelay's documentation

The result is a network which behaves identically, except that outputs are produced n timesteps later.

it becomes clear that you're not changing the network, instead you only change the time dependence of your y-values, so your network will try to predict one time step ahead. You can see this behaviour at the very end of your T and Y vectors where Y will have an additional value while T fills this space with NaN (because you obviously cannot generate more targets out of the blue).

removedelay() is supposed to be used in combination with a closed loop design, so that you can obtain predicted values early to use them as direct input for the next step. In this case, it also makes sense to increase the output delay by more than just one which is why you can pass an additional argument n:

net = removedelay(net,n);

To prove that the additional time step is not used you can simulate the desired data set with your trained net and then simulate the same set with removedelay(). They are going to be identical except for the last value of the Y curve (see Figure 1).

Fig. 1: Both plots are based on the same net trained with the first 3500 data points of MATLAB's heat exchanger example. Shown are the simulation results for the last 500 values in the set that have not been used in the training process. The results are identical except for an additional value for the one on the left using removedelay().

Errors

Your errors have to be very small if you're using a representative training set. Therefore, the prediction for similar, new data will be good because your net is not overfitted.

Conclusions

So, are you predicting? No, you are simulating. Simulating your network's behaviour is based on inputs of your previously unknown data set, not the targets (they only have to be passed to allow for performance evaluation). Therefore, passing new data to your net with or without removedelay() is simulation in both cases because it is based on provided inputs. Removing the delay doesn't make a difference for these results.
Prediction, on the other hand, requires no input data because it really just continues the pattern the network has learned so far without taking new input into account.

Suggestions

If all you want is to have an unknown data set with valid input values passed to your net for simulation, you could just as well pass it as part of the testing set by using the divideblock or divideint options.

If you want to make use of early prediction by removedelay() or need prediction in general because your inputs have holes or are unreliable for other reasons, you should consider simulating your unknown set with a closed loop. Should its performance be all too awful you can also train a closed loop network from the very beginning.