Delphi: EInvalidOp in neural network class (TD-lambda)

Question

I have the following draft for a neural network class. This neural network should learn with TD-lambda. It is started by calling the getRating() function.

But unfortunately, there is an EInvalidOp (invalid floading point operation) error after about 1000 iterations in the following lines:

neuronsHidden[j] := neuronsHidden[j]+neuronsInput[t][i]*weightsInput[i][j]; // input -> hidden

weightsHidden[j][k] := weightsHidden[j][k]+LEARNING_RATE_HIDDEN*tdError[k]*eligibilityTraceOutput[j][k]; // adjust hidden->output weights according to TD-lambda

Why is this error? I can't find the mistake in my code :( Can you help me? Thank you very much in advance!

learningMode: Boolean; // does the network learn and change its weights?
neuronsInput: Array[1..MAX_TIMESTEPS] of Array[1..NEURONS_INPUT] of Extended;
neuronsHidden: Array[1..NEURONS_HIDDEN] of Extended;
neuronsOutput: Array[1..NEURONS_OUTPUT] of Extended;
weightsInput: Array[1..NEURONS_INPUT] of Array[1..NEURONS_HIDDEN] of Extended;
weightsHidden: Array[1..NEURONS_HIDDEN] of Array[1..NEURONS_OUTPUT] of Extended;

[...]

function HyperbolicTangent;
begin
  if x > 5500 then // prevent overflow
    result := 1
  else
    result := (Exp(2*x)-1)/(Exp(2*x)+1);
end;
[...]

Answer 1

The most likely reason for you error is that you are overflowing the accumulators neuronsHidden or weightsHidden. I know nothing about neural networks so I can't offer any explanation as to why this is so.

As a side issue, I question the use of Extended floating point variables. Normally this simply results in extremely slow performance, much slower than when using Double. You may think it gives you more headroom for large numbers, but in reality, if this is a run-away overflow of the nature that I suspect, then using Extended would never save you.

UPDATE OP points out that overflow results in a different exception class. So for EInvalidOp I suspect some square root or trig failure or something like that. Or perhaps a signaling NaN, but since you don't obviously use uninitialised data I'll not persue that.

I can see now that you have been affected by Embarcadero's bizarre decision to break their implementation of Tanh. This used to work perfectly on older versions of Delphi (e.g. D6), but was broken recently. The version you use isn't quite right for large negative input, and uses two calls to Exp when one suffices. I use this version:

const
  MaxTanhDomain = 5678.22249441322; // Ln(MaxExtended)/2

function Tanh(const X: Extended): Extended;
begin
  if X>MaxTanhDomain then begin
    Result := 1.0
  end else if X<-MaxTanhDomain then begin
    Result := -1.0
  end else begin
    Result := Exp(X);
    Result := Result*Result;
    Result := (Result-1.0)/(Result+1.0);
  end;
end;

Answer 2

Not an answer, just a suggestion; The two lines of code you're showing only include multiplication and addition, very simple operations. How about logging the values on failure, maybe seeing the values you can figure something out.

The most annoying trouble with stopping on exception is that you can't inspect the variables involved in that exception. To work around that limitation I sometimes wrap the troublesome operation in an try-except block and place a breakpoint in the except handler; Delphi will first stop at the exception, I hit run, and then it stops on my breakpoint. At the breakpoint I can freely inspect all variables used in the error-generating statement, so I can figure out what's wrong.

// In place of:
neuronsHidden[j] := neuronsHidden[j]+neuronsInput[t][i]*weightsInput[i][j];

var saveNerusonsHidden: Double;
try
  saveNeuronsHidden := neuronsHidden[j]; // saved, to be sure I can inspect the original value
  neuronsHidden[j] := neuronsHidden[j]+neuronsInput[t][i]*weightsInput[i][j];
except on E:EInvalidOp do
  begin
    // Breakpoint here, so you can inspect the values of neuronsInput[t][i], wightsInput[i][j] and saveNeuronsHidden
    raise;
  end;
end;

Answer 3

You can get EInvalidOp when calculation very large or very small numbers.

When getting this error, maybe you can debug/view the values in the array and do a partial calculation in the watch list?